2024-09-17T22:46:44.095 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T22:46:44.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T22:46:44.191 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909554' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBP3k32Kst8tQSHniPqTT7yleeKRhKtdF1o1YlGImb9Z0J8kHuwpVSfj0vCjhajpLd6ySYyIt0gQkIEt6m9hApw= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHc2LG74uPPb3WAok5lbxqcDxRG3Oa26xCeBMmquyrnFuqq9MjLOGNXm6/UbWFZC3vP+iBh1dJcs3odAcURWrc= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T22:46:44.192 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-17T22:46:44.192 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-17T22:46:44.193 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T22:46:44.194 INFO:teuthology.task.internal:Checking packages... 2024-09-17T22:46:44.213 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-17T22:46:44.214 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T22:46:44.214 INFO:teuthology.packaging:ref: None 2024-09-17T22:46:44.214 INFO:teuthology.packaging:tag: None 2024-09-17T22:46:44.214 INFO:teuthology.packaging:branch: squid 2024-09-17T22:46:44.214 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:46:44.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-17T22:46:44.433 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-17T22:46:44.435 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T22:46:44.442 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T22:46:44.442 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T22:46:44.481 INFO:teuthology.task.internal:Saving configuration 2024-09-17T22:46:44.494 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T22:46:44.513 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T22:46:44.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 22:41:38.566406', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBP3k32Kst8tQSHniPqTT7yleeKRhKtdF1o1YlGImb9Z0J8kHuwpVSfj0vCjhajpLd6ySYyIt0gQkIEt6m9hApw='} 2024-09-17T22:46:44.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 22:41:38.568378', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHc2LG74uPPb3WAok5lbxqcDxRG3Oa26xCeBMmquyrnFuqq9MjLOGNXm6/UbWFZC3vP+iBh1dJcs3odAcURWrc='} 2024-09-17T22:46:44.561 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T22:46:44.569 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-17T22:46:44.569 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-17T22:46:44.569 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T22:46:44.642 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f238768fb50>, signals=[15]) 2024-09-17T22:46:44.642 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T22:46:44.650 INFO:teuthology.task.internal:Opening connections... 2024-09-17T22:46:44.650 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:46:44.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:46:44.725 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:46:44.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:46:44.798 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T22:46:44.805 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-09-17T22:46:44.823 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-09-17T22:46:44.824 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-09-17T22:46:44.882 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T22:46:44.883 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T22:46:44.883 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T22:46:44.883 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-09-17T22:46:44.975 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-17T22:46:44.994 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-17T22:46:44.994 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-17T22:46:45.052 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T22:46:45.053 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T22:46:45.053 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-17T22:46:45.083 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T22:46:45.093 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T22:46:45.177 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T22:46:45.177 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-09-17T22:46:45.179 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-17T22:46:45.196 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T22:46:45.205 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T22:46:45.205 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-09-17T22:46:45.236 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-17T22:46:45.336 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T22:46:45.401 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T22:46:45.417 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T22:46:45.417 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T22:46:45.417 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T22:46:45.418 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T22:46:45.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:46:45.435 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-09-17T22:46:45.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:46:45.436 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-17T22:46:45.493 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-503.el9.x86_64 2024-09-17T22:46:45.493 INFO:teuthology.task.kernel:Running kernel on smithi159: 5.14.0-503.el9.x86_64 2024-09-17T22:46:45.493 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum install -y kernel 2024-09-17T22:46:45.494 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-503.el9.x86_64 2024-09-17T22:46:45.494 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-503.el9.x86_64 2024-09-17T22:46:45.494 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-09-17T22:46:48.112 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-17T22:46:49.120 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-17T22:46:49.946 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-17T22:46:50.993 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T22:46:55.022 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.5 MB 00:01 2024-09-17T22:46:55.344 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-17T22:46:56.778 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-17T22:46:57.348 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-17T22:46:59.485 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:02 2024-09-17T22:46:59.590 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 23 MB 00:02 2024-09-17T22:47:04.916 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T22:47:04.937 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T22:47:06.789 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T22:47:06.789 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T22:47:06.835 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T22:47:06.836 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T22:47:06.867 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T22:47:06.873 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout:Install 4 Packages 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:47:06.874 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 79 M 2024-09-17T22:47:06.875 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 126 M 2024-09-17T22:47:06.875 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:47:06.922 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T22:47:06.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 79 M 2024-09-17T22:47:06.930 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 126 M 2024-09-17T22:47:06.931 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:47:06.975 INFO:teuthology.orchestra.run.smithi159.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 446 kB/s | 36 kB 00:00 2024-09-17T22:47:07.072 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 387 kB/s | 36 kB 00:00 2024-09-17T22:47:07.359 INFO:teuthology.orchestra.run.smithi159.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-17T22:47:07.843 INFO:teuthology.orchestra.run.smithi159.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-17T22:47:08.135 INFO:teuthology.orchestra.run.smithi159.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-17T22:47:08.135 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:47:08.135 INFO:teuthology.orchestra.run.smithi159.stdout:Total 63 MB/s | 79 MB 00:01 2024-09-17T22:47:08.815 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:47:09.169 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:47:09.170 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:47:09.831 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-09-17T22:47:10.127 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:47:10.127 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:47:10.947 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 7.4 MB/s | 29 MB 00:03 2024-09-17T22:47:11.481 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 7.7 MB/s | 35 MB 00:04 2024-09-17T22:47:11.481 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:47:11.481 INFO:teuthology.orchestra.run.smithi171.stdout:Total 17 MB/s | 79 MB 00:04 2024-09-17T22:47:12.164 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:47:12.532 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:47:12.532 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:47:12.537 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:47:13.205 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:47:13.285 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:47:13.491 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:47:13.491 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:47:15.944 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:47:16.057 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:47:16.207 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:47:16.594 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:47:16.691 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:47:19.450 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:47:19.583 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:47:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:47:21.090 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:47:24.448 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:47:24.776 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:47:25.913 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:47:29.597 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:04.307 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:04.667 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:04.667 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:48:04.668 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:48:04.668 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.484 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.485 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.485 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:48:05.485 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:48:05.814 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:48:05.909 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.910 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:48:05.910 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T22:48:05.910 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T22:48:05.910 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T22:48:05.910 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T22:48:05.910 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T22:48:05.910 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:48:05.969 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:05.970 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:48:05.970 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.06 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 82.el9 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Group : Unspecified 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 8274670 2024-09-17T22:48:06.008 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : builder@centos.org 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T22:48:06.009 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T22:48:06.010 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T22:48:06.010 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2024-09-17T22:48:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:48:06.010 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T22:48:06.010 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T22:48:06.010 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 5.14.0-508.el9.x86_64 2024-09-17T22:48:06.011 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T22:48:06.583 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2024-09-17T22:48:07.835 INFO:teuthology.orchestra.run.smithi159.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T22:48:07.838 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:07.872 INFO:teuthology.orchestra.run.smithi159.stderr:done 2024-09-17T22:48:07.874 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-09-17T22:48:07.879 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:07.890 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.7LmUqVBJ1s 2024-09-17T22:48:07.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7LmUqVBJ1s 2024-09-17T22:48:07.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.7LmUqVBJ1s 2024-09-17T22:48:08.102 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.7LmUqVBJ1s is 6KB 2024-09-17T22:48:08.151 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.7LmUqVBJ1s 2024-09-17T22:48:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:48:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:48:08.156 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:48:08.166 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T22:48:08.229 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T22:48:08.229 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T22:48:08.229 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T22:48:08.229 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T22:48:08.231 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T22:48:08.391 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2024-09-17T22:48:09.030 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:48:09.030 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:48:09.030 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:48:09.030 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.031 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.031 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.031 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.031 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:48:09.031 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:48:09.402 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:48:09.486 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.486 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:48:09.486 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T22:48:09.486 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T22:48:09.487 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T22:48:09.487 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T22:48:09.487 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T22:48:09.487 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:48:09.546 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:48:09.547 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:48:09.547 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 82.el9 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8274670 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-09-17T22:48:09.582 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-09-17T22:48:09.583 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:48:09.584 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T22:48:09.584 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T22:48:09.585 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 5.14.0-508.el9.x86_64 2024-09-17T22:48:09.585 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T22:48:10.280 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-17T22:48:11.579 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T22:48:11.616 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-17T22:48:11.618 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-17T22:48:11.634 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.Nue6stpqJJ 2024-09-17T22:48:11.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Nue6stpqJJ 2024-09-17T22:48:11.703 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.Nue6stpqJJ 2024-09-17T22:48:11.836 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.Nue6stpqJJ is 6KB 2024-09-17T22:48:11.889 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.Nue6stpqJJ 2024-09-17T22:48:11.904 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T22:48:11.972 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T22:48:11.972 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T22:48:11.972 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T22:48:11.972 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T22:48:11.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T22:48:12.132 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-17T22:48:38.417 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T22:48:38.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:48:38.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:48:42.137 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T22:48:42.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:48:42.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:48:56.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-09-17T22:49:00.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-17T22:49:05.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:49:05.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:08.957 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-09-17T22:49:09.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:49:09.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:12.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-17T22:49:20.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:49:20.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:24.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:49:24.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:39.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-09-17T22:49:42.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-17T22:49:54.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:49:54.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:54.744 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-17T22:49:55.133 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:49:55.133 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T22:49:55.134 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-09-17T22:49:55.148 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-508.el9.x86_64 2024-09-17T22:49:55.148 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T22:49:55.148 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T22:49:55.148 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T22:49:56.149 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T22:49:56.149 DEBUG:teuthology.orchestra.run.smithi159:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T22:49:56.236 INFO:teuthology.orchestra.run.smithi159.stdout:ttyS1 2024-09-17T22:49:56.256 DEBUG:teuthology.parallel:result is None 2024-09-17T22:49:57.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:49:57.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:49:58.329 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-17T22:49:58.750 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:49:58.750 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T22:49:58.751 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-17T22:49:58.765 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-508.el9.x86_64 2024-09-17T22:49:58.765 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T22:49:58.765 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T22:49:58.765 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T22:49:59.766 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T22:49:59.766 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T22:50:00.003 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-17T22:50:00.022 DEBUG:teuthology.parallel:result is None 2024-09-17T22:50:00.022 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T22:50:00.031 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T22:50:00.031 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T22:50:00.034 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T22:50:00.049 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T22:50:00.069 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T22:50:00.100 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T22:50:00.100 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T22:50:00.102 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T22:50:00.148 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T22:50:00.156 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T22:50:00.156 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T22:50:00.160 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T22:50:00.187 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:50:00.201 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:50:00.403 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:50:00.420 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:50:00.422 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T22:50:00.430 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T22:50:00.431 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T22:50:00.433 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T22:50:00.499 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T22:50:00.510 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T22:50:00.510 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T22:50:00.512 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T22:50:00.555 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:50:00.609 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:50:00.653 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:50:00.732 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:50:00.778 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:50:00.778 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T22:50:00.844 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:50:00.911 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:50:00.990 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:50:01.057 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:50:01.140 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:50:01.140 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T22:50:01.165 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-09-17T22:50:01.167 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-17T22:50:01.248 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:50:01.273 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:50:01.616 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T22:50:01.624 INFO:teuthology.task.internal:Starting timer... 2024-09-17T22:50:01.624 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T22:50:01.670 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T22:50:01.704 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-17T22:50:01.705 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-09-17T22:50:01.833 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-09-17T22:50:01.836 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-17T22:50:01.969 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-17T22:50:01.971 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T22:50:01.971 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-09-17T22:50:02.041 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-09-17T22:50:02.041 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-17T22:50:02.113 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-17T22:50:02.113 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-09-17T22:50:02.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:50:02.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:50:02.196 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:50:02.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:50:02.224 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T22:50:02.224 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-09-17T22:50:02.372 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-17T22:50:02.403 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T22:50:02.419 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T22:50:02.429 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T22:50:02.430 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi159.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T22:53:45.652 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-09-17T22:53:45.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:53:45.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:53:45.732 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-17T22:53:45.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-17T22:53:45.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:53:45.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:53:45.897 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-17T22:53:45.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-17T22:53:45.981 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T22:53:45.992 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T22:53:45.993 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T22:53:45.993 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:53:45.996 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T22:53:45.996 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:53:46.023 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T22:53:46.036 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T22:53:46.054 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T22:53:46.061 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-09-17T22:53:46.069 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T22:53:46.072 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-09-17T22:53:46.086 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T22:53:46.095 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-17T22:53:46.100 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T22:53:46.106 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-17T22:53:46.121 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T22:53:46.136 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T22:53:46.157 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:53:46.160 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:53:46.160 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-17T22:53:46.160 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.160 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.161 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.161 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.196 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.200 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:53:46.200 INFO:teuthology.run_tasks:Running task install... 2024-09-17T22:53:46.209 DEBUG:teuthology.task.install:project ceph 2024-09-17T22:53:46.209 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-17T22:53:46.209 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-17T22:53:46.209 INFO:teuthology.task.install:Using flavor: default 2024-09-17T22:53:46.216 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T22:53:46.216 INFO:teuthology.task.install:extra packages: [] 2024-09-17T22:53:46.216 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T22:53:46.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:53:46.219 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T22:53:46.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:53:46.362 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T22:53:46.362 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T22:53:46.365 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T22:53:46.365 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T22:53:46.474 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T22:53:46.475 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:53:46.475 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T22:53:46.481 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T22:53:46.481 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:53:46.482 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T22:53:46.501 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T22:53:46.501 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T22:53:46.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T22:53:46.508 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T22:53:46.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T22:53:46.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T22:53:46.645 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T22:53:46.649 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T22:53:46.709 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-09-17T22:53:46.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-09-17T22:53:46.713 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-17T22:53:46.715 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-17T22:53:46.940 INFO:teuthology.orchestra.run.smithi159.stdout:45 files removed 2024-09-17T22:53:46.947 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-17T22:53:46.967 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw 2024-09-17T22:53:46.974 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw 2024-09-17T22:53:47.511 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-17T22:53:47.526 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-17T22:53:47.795 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T22:53:47.811 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T22:53:48.064 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T22:53:48.086 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T22:53:48.770 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-17T22:53:49.739 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-17T22:53:52.014 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 20 MB 00:02 2024-09-17T22:53:53.041 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 20 MB 00:02 2024-09-17T22:53:56.265 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T22:53:57.691 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 19 kB 00:00 2024-09-17T22:53:58.783 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-17T22:53:59.362 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-17T22:54:01.108 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-17T22:54:03.988 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 23 MB 00:02 2024-09-17T22:54:04.718 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T22:54:06.936 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-17T22:54:06.943 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T22:54:06.944 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T22:54:06.945 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout:Install 30 Packages 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 2 Packages 2024-09-17T22:54:06.946 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:06.947 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 60 M 2024-09-17T22:54:06.947 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:54:07.604 INFO:teuthology.orchestra.run.smithi171.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 12 MB/s | 5.4 MB 00:00 2024-09-17T22:54:07.646 INFO:teuthology.orchestra.run.smithi171.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 607 kB/s | 25 kB 00:00 2024-09-17T22:54:07.738 INFO:teuthology.orchestra.run.smithi171.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.0 MB/s | 751 kB 00:00 2024-09-17T22:54:07.830 INFO:teuthology.orchestra.run.smithi171.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 5.4 MB/s | 506 kB 00:00 2024-09-17T22:54:08.138 INFO:teuthology.orchestra.run.smithi171.stdout:(5/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 11 MB/s | 11 MB 00:00 2024-09-17T22:54:08.239 INFO:teuthology.orchestra.run.smithi171.stdout:(6/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 13 MB/s | 5.3 MB 00:00 2024-09-17T22:54:08.264 INFO:teuthology.orchestra.run.smithi171.stdout:(7/32): python3-ceph-argparse-19.1.1-330.gc2ddf 360 kB/s | 45 kB 00:00 2024-09-17T22:54:08.298 INFO:teuthology.orchestra.run.smithi171.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.3 MB/s | 136 kB 00:00 2024-09-17T22:54:08.323 INFO:teuthology.orchestra.run.smithi171.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 2.7 MB/s | 162 kB 00:00 2024-09-17T22:54:08.349 INFO:teuthology.orchestra.run.smithi171.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.2 MB/s | 320 kB 00:00 2024-09-17T22:54:08.557 INFO:teuthology.orchestra.run.smithi171.stdout:(11/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 15 MB/s | 21 MB 00:01 2024-09-17T22:54:08.582 INFO:teuthology.orchestra.run.smithi171.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 425 kB/s | 99 kB 00:00 2024-09-17T22:54:08.608 INFO:teuthology.orchestra.run.smithi171.stdout:(13/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 1.0 MB/s | 300 kB 00:00 2024-09-17T22:54:08.684 INFO:teuthology.orchestra.run.smithi171.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 850 kB/s | 107 kB 00:00 2024-09-17T22:54:08.709 INFO:teuthology.orchestra.run.smithi171.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-17T22:54:08.751 INFO:teuthology.orchestra.run.smithi171.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 955 kB/s | 160 kB 00:00 2024-09-17T22:54:08.793 INFO:teuthology.orchestra.run.smithi171.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T22:54:08.818 INFO:teuthology.orchestra.run.smithi171.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T22:54:08.852 INFO:teuthology.orchestra.run.smithi171.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.8 MB/s | 662 kB 00:00 2024-09-17T22:54:08.869 INFO:teuthology.orchestra.run.smithi171.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 712 kB/s | 36 kB 00:00 2024-09-17T22:54:08.886 INFO:teuthology.orchestra.run.smithi171.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-17T22:54:08.911 INFO:teuthology.orchestra.run.smithi171.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-17T22:54:08.970 INFO:teuthology.orchestra.run.smithi171.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 599 kB/s | 26 kB 00:00 2024-09-17T22:54:08.996 INFO:teuthology.orchestra.run.smithi171.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.4 MB/s | 308 kB 00:00 2024-09-17T22:54:09.013 INFO:teuthology.orchestra.run.smithi171.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-17T22:54:09.038 INFO:teuthology.orchestra.run.smithi171.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-17T22:54:09.089 INFO:teuthology.orchestra.run.smithi171.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-17T22:54:09.206 INFO:teuthology.orchestra.run.smithi171.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-09-17T22:54:09.373 INFO:teuthology.orchestra.run.smithi171.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 21 MB/s | 3.4 MB 00:00 2024-09-17T22:54:09.393 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-17T22:54:09.415 INFO:teuthology.orchestra.run.smithi171.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-17T22:54:09.515 INFO:teuthology.orchestra.run.smithi171.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 22 MB/s | 3.2 MB 00:00 2024-09-17T22:54:09.607 INFO:teuthology.orchestra.run.smithi171.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-09-17T22:54:09.611 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:09.611 INFO:teuthology.orchestra.run.smithi171.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-17T22:54:09.714 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:54:09.750 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:54:09.751 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:54:10.440 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:54:10.440 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:54:11.587 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:54:11.600 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:11.600 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T22:54:11.601 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T22:54:11.602 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T22:54:11.603 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:Install 30 Packages 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 2 Packages 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 60 M 2024-09-17T22:54:11.604 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:54:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:54:12.058 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T22:54:12.087 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T22:54:12.113 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T22:54:12.136 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:54:12.157 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:54:12.346 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:54:12.399 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:54:12.586 INFO:teuthology.orchestra.run.smithi159.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 10 MB/s | 5.4 MB 00:00 2024-09-17T22:54:12.636 INFO:teuthology.orchestra.run.smithi159.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 504 kB/s | 25 kB 00:00 2024-09-17T22:54:12.737 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T22:54:12.744 INFO:teuthology.orchestra.run.smithi159.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 6.8 MB/s | 751 kB 00:00 2024-09-17T22:54:12.845 INFO:teuthology.orchestra.run.smithi159.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 4.9 MB/s | 506 kB 00:00 2024-09-17T22:54:12.877 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:13.067 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T22:54:13.078 INFO:teuthology.orchestra.run.smithi159.stdout:(5/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 21 MB/s | 21 MB 00:01 2024-09-17T22:54:13.093 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:54:13.148 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:54:13.185 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T22:54:13.195 INFO:teuthology.orchestra.run.smithi159.stdout:(6/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 9.3 MB/s | 11 MB 00:01 2024-09-17T22:54:13.230 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T22:54:13.239 INFO:teuthology.orchestra.run.smithi159.stdout:(7/32): python3-ceph-argparse-19.1.1-330.gc2ddf 282 kB/s | 45 kB 00:00 2024-09-17T22:54:13.263 INFO:teuthology.orchestra.run.smithi159.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.0 MB/s | 136 kB 00:00 2024-09-17T22:54:13.288 INFO:teuthology.orchestra.run.smithi159.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 3.2 MB/s | 162 kB 00:00 2024-09-17T22:54:13.313 INFO:teuthology.orchestra.run.smithi159.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.2 MB/s | 320 kB 00:00 2024-09-17T22:54:13.317 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:54:13.338 INFO:teuthology.orchestra.run.smithi159.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-17T22:54:13.354 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:13.364 INFO:teuthology.orchestra.run.smithi159.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T22:54:13.419 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:13.447 INFO:teuthology.orchestra.run.smithi159.stdout:(13/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 8.9 MB/s | 5.3 MB 00:00 2024-09-17T22:54:13.462 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T22:54:13.489 INFO:teuthology.orchestra.run.smithi159.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 711 kB/s | 107 kB 00:00 2024-09-17T22:54:13.539 INFO:teuthology.orchestra.run.smithi159.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 494 kB/s | 45 kB 00:00 2024-09-17T22:54:13.565 INFO:teuthology.orchestra.run.smithi159.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 799 kB/s | 160 kB 00:00 2024-09-17T22:54:13.581 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T22:54:13.590 INFO:teuthology.orchestra.run.smithi159.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-17T22:54:13.613 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T22:54:13.615 INFO:teuthology.orchestra.run.smithi159.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 842 kB/s | 42 kB 00:00 2024-09-17T22:54:13.640 INFO:teuthology.orchestra.run.smithi159.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-17T22:54:13.659 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T22:54:13.666 INFO:teuthology.orchestra.run.smithi159.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-17T22:54:13.687 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T22:54:13.692 INFO:teuthology.orchestra.run.smithi159.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-17T22:54:13.716 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T22:54:13.716 INFO:teuthology.orchestra.run.smithi159.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-17T22:54:13.750 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T22:54:13.783 INFO:teuthology.orchestra.run.smithi159.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 504 kB/s | 26 kB 00:00 2024-09-17T22:54:13.809 INFO:teuthology.orchestra.run.smithi159.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-17T22:54:13.834 INFO:teuthology.orchestra.run.smithi159.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-17T22:54:13.859 INFO:teuthology.orchestra.run.smithi159.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T22:54:13.918 INFO:teuthology.orchestra.run.smithi159.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-17T22:54:14.043 INFO:teuthology.orchestra.run.smithi159.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T22:54:14.121 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:54:14.212 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:54:14.235 INFO:teuthology.orchestra.run.smithi159.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 18 MB/s | 3.4 MB 00:00 2024-09-17T22:54:14.242 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T22:54:14.277 INFO:teuthology.orchestra.run.smithi159.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-09-17T22:54:14.295 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T22:54:14.394 INFO:teuthology.orchestra.run.smithi159.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 20 MB/s | 3.2 MB 00:00 2024-09-17T22:54:14.461 INFO:teuthology.orchestra.run.smithi159.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-09-17T22:54:14.465 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:14.466 INFO:teuthology.orchestra.run.smithi159.stdout:Total 21 MB/s | 60 MB 00:02 2024-09-17T22:54:14.571 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:54:14.608 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:54:14.608 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:54:14.792 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T22:54:14.824 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:54:14.883 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:54:14.956 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T22:54:14.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T22:54:15.243 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:54:15.243 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:54:16.753 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:54:16.836 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T22:54:16.866 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T22:54:16.891 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T22:54:16.913 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:54:16.936 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:54:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:54:17.177 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:54:17.354 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:17.391 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:17.516 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T22:54:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:17.719 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:17.806 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T22:54:17.836 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:54:17.899 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:54:17.936 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T22:54:17.960 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T22:54:18.004 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:54:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:54:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:54:18.110 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T22:54:18.110 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T22:54:18.110 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:18.147 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T22:54:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T22:54:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T22:54:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T22:54:18.394 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T22:54:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T22:54:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T22:54:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T22:54:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:54:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:54:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T22:54:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T22:54:19.491 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T22:54:19.521 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:54:19.588 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:54:19.661 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T22:54:19.690 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T22:54:21.957 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:22.577 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:54:22.740 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:54:22.839 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:54:22.839 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T22:54:22.839 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T22:54:22.839 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:22.874 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:28.213 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-17T22:54:28.214 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:29.205 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:54:29.230 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:54:29.230 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:29.230 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T22:54:29.230 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:54:29.231 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:54:29.231 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:29.232 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:54:29.316 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:54:29.316 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:30.574 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T22:54:30.575 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T22:54:30.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T22:54:30.577 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T22:54:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T22:54:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T22:54:30.578 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T22:54:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T22:54:30.579 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T22:54:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:31.695 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T22:54:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:31.698 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:54:32.041 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-test 2024-09-17T22:54:32.614 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:54:32.749 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 49 M 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 209 M 2024-09-17T22:54:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:54:32.817 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-09-17T22:54:32.818 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:32.992 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 743 kB/s | 163 kB 00:00 2024-09-17T22:54:33.568 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 382 kB/s | 304 kB 00:00 2024-09-17T22:54:33.609 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 104 kB/s | 64 kB 00:00 2024-09-17T22:54:33.802 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:54:33.827 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:33.829 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:54:33.912 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:54:33.912 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:34.268 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 33 MB/s | 49 MB 00:01 2024-09-17T22:54:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:34.271 INFO:teuthology.orchestra.run.smithi171.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-17T22:54:34.299 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:54:34.320 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:54:34.320 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:54:34.705 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:54:34.705 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T22:54:35.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T22:54:35.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T22:54:35.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T22:54:35.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:54:35.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T22:54:35.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T22:54:35.250 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:54:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T22:54:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T22:54:35.396 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.013 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T22:54:36.014 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:36.015 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:54:36.356 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-test 2024-09-17T22:54:36.935 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:54:37.066 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T22:54:37.067 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout:Install 4 Packages 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:37.068 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 49 M 2024-09-17T22:54:37.069 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 209 M 2024-09-17T22:54:37.069 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:54:37.325 INFO:teuthology.orchestra.run.smithi159.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-17T22:54:37.367 INFO:teuthology.orchestra.run.smithi159.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T22:54:37.517 INFO:teuthology.orchestra.run.smithi159.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 469 kB/s | 163 kB 00:00 2024-09-17T22:54:38.818 INFO:teuthology.orchestra.run.smithi159.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 49 MB 00:01 2024-09-17T22:54:38.820 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:38.820 INFO:teuthology.orchestra.run.smithi159.stdout:Total 28 MB/s | 49 MB 00:01 2024-09-17T22:54:38.848 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:54:38.869 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:54:38.869 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:54:39.255 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:54:39.256 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:54:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:54:39.862 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T22:54:39.901 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T22:54:39.928 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:54:40.025 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:54:40.104 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:54:42.543 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:54:42.544 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T22:54:42.544 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T22:54:42.544 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:42.859 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:54:43.152 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph 2024-09-17T22:54:43.747 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:54:43.888 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:43.890 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:54:43.890 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:43.890 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:54:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T22:54:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T22:54:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T22:54:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout:Install 37 Packages 2024-09-17T22:54:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:43.895 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 30 M 2024-09-17T22:54:43.895 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 108 M 2024-09-17T22:54:43.895 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:54:44.267 INFO:teuthology.orchestra.run.smithi171.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 47 kB/s | 6.5 kB 00:00 2024-09-17T22:54:44.434 INFO:teuthology.orchestra.run.smithi171.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-17T22:54:44.493 INFO:teuthology.orchestra.run.smithi171.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-17T22:54:44.552 INFO:teuthology.orchestra.run.smithi171.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 4.2 MB/s | 249 kB 00:00 2024-09-17T22:54:44.592 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:54:44.677 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:54:44.727 INFO:teuthology.orchestra.run.smithi171.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-17T22:54:44.987 INFO:teuthology.orchestra.run.smithi171.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-17T22:54:45.020 INFO:teuthology.orchestra.run.smithi171.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 4.2 MB/s | 1.2 MB 00:00 2024-09-17T22:54:45.095 INFO:teuthology.orchestra.run.smithi171.stdout:(8/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 5.7 MB/s | 4.7 MB 00:00 2024-09-17T22:54:45.120 INFO:teuthology.orchestra.run.smithi171.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.3 MB/s | 135 kB 00:00 2024-09-17T22:54:45.146 INFO:teuthology.orchestra.run.smithi171.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 669 kB/s | 106 kB 00:00 2024-09-17T22:54:45.172 INFO:teuthology.orchestra.run.smithi171.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.1 MB/s | 215 kB 00:00 2024-09-17T22:54:45.222 INFO:teuthology.orchestra.run.smithi171.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-17T22:54:45.247 INFO:teuthology.orchestra.run.smithi171.stdout:(13/37): python3-requests-2.25.1-8.el9.noarch.r 825 kB/s | 125 kB 00:00 2024-09-17T22:54:45.273 INFO:teuthology.orchestra.run.smithi171.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-17T22:54:45.298 INFO:teuthology.orchestra.run.smithi171.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 459 kB/s | 35 kB 00:00 2024-09-17T22:54:45.323 INFO:teuthology.orchestra.run.smithi171.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-17T22:54:45.348 INFO:teuthology.orchestra.run.smithi171.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-17T22:54:45.374 INFO:teuthology.orchestra.run.smithi171.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 672 kB/s | 51 kB 00:00 2024-09-17T22:54:45.432 INFO:teuthology.orchestra.run.smithi171.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 746 kB/s | 43 kB 00:00 2024-09-17T22:54:45.458 INFO:teuthology.orchestra.run.smithi171.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-09-17T22:54:45.491 INFO:teuthology.orchestra.run.smithi171.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-09-17T22:54:45.525 INFO:teuthology.orchestra.run.smithi171.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 403 kB/s | 11 kB 00:00 2024-09-17T22:54:45.559 INFO:teuthology.orchestra.run.smithi171.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-17T22:54:45.585 INFO:teuthology.orchestra.run.smithi171.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 309 kB/s | 18 kB 00:00 2024-09-17T22:54:45.618 INFO:teuthology.orchestra.run.smithi171.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-17T22:54:45.640 INFO:teuthology.orchestra.run.smithi171.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 354 kB/s | 19 kB 00:00 2024-09-17T22:54:45.668 INFO:teuthology.orchestra.run.smithi171.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 421 kB/s | 20 kB 00:00 2024-09-17T22:54:45.690 INFO:teuthology.orchestra.run.smithi171.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-17T22:54:45.716 INFO:teuthology.orchestra.run.smithi171.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T22:54:45.741 INFO:teuthology.orchestra.run.smithi171.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-17T22:54:45.766 INFO:teuthology.orchestra.run.smithi171.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T22:54:45.792 INFO:teuthology.orchestra.run.smithi171.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T22:54:45.817 INFO:teuthology.orchestra.run.smithi171.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-17T22:54:45.851 INFO:teuthology.orchestra.run.smithi171.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T22:54:45.876 INFO:teuthology.orchestra.run.smithi171.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 803 kB/s | 20 kB 00:00 2024-09-17T22:54:45.902 INFO:teuthology.orchestra.run.smithi171.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-17T22:54:45.935 INFO:teuthology.orchestra.run.smithi171.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-17T22:54:45.939 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:45.939 INFO:teuthology.orchestra.run.smithi171.stdout:Total 15 MB/s | 30 MB 00:02 2024-09-17T22:54:46.090 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:54:46.131 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:54:46.131 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:54:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:54:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:54:47.213 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:54:47.213 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T22:54:47.213 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T22:54:47.213 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:47.517 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T22:54:47.518 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T22:54:47.518 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:47.518 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:54:47.784 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph 2024-09-17T22:54:48.031 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:54:48.089 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T22:54:48.134 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T22:54:48.175 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T22:54:48.247 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T22:54:48.299 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T22:54:48.333 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T22:54:48.382 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:54:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T22:54:48.503 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T22:54:48.527 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T22:54:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T22:54:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T22:54:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T22:54:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T22:54:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T22:54:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T22:54:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T22:54:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:48.536 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout:Install 37 Packages 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 30 M 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 108 M 2024-09-17T22:54:48.537 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:54:48.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T22:54:48.588 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T22:54:48.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T22:54:48.652 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T22:54:48.686 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T22:54:48.735 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T22:54:48.770 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T22:54:48.813 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T22:54:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T22:54:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T22:54:48.975 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T22:54:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T22:54:49.161 INFO:teuthology.orchestra.run.smithi159.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-17T22:54:49.204 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T22:54:49.265 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T22:54:49.312 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T22:54:49.361 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T22:54:49.378 INFO:teuthology.orchestra.run.smithi159.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.5 MB/s | 1.5 MB 00:00 2024-09-17T22:54:49.417 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T22:54:49.436 INFO:teuthology.orchestra.run.smithi159.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.0 MB/s | 2.4 MB 00:00 2024-09-17T22:54:49.520 INFO:teuthology.orchestra.run.smithi159.stdout:(4/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-17T22:54:49.545 INFO:teuthology.orchestra.run.smithi159.stdout:(5/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.2 MB/s | 249 kB 00:00 2024-09-17T22:54:49.602 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T22:54:49.651 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T22:54:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T22:54:49.810 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T22:54:49.848 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T22:54:49.921 INFO:teuthology.orchestra.run.smithi159.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-17T22:54:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T22:54:50.132 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T22:54:50.170 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:54:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:50.238 INFO:teuthology.orchestra.run.smithi159.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 353 kB/s | 253 kB 00:00 2024-09-17T22:54:50.346 INFO:teuthology.orchestra.run.smithi159.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.2 MB/s | 135 kB 00:00 2024-09-17T22:54:50.371 INFO:teuthology.orchestra.run.smithi159.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 236 kB/s | 106 kB 00:00 2024-09-17T22:54:50.463 INFO:teuthology.orchestra.run.smithi159.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.0 MB/s | 125 kB 00:00 2024-09-17T22:54:50.580 INFO:teuthology.orchestra.run.smithi159.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-17T22:54:50.689 INFO:teuthology.orchestra.run.smithi159.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 677 kB/s | 215 kB 00:00 2024-09-17T22:54:51.256 INFO:teuthology.orchestra.run.smithi159.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 62 kB/s | 35 kB 00:00 2024-09-17T22:54:51.306 INFO:teuthology.orchestra.run.smithi159.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 224 kB/s | 188 kB 00:00 2024-09-17T22:54:51.356 INFO:teuthology.orchestra.run.smithi159.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-09-17T22:54:51.639 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:54:51.666 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:54:51.666 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:51.666 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T22:54:51.666 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:54:51.666 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:54:51.667 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:51.940 INFO:teuthology.orchestra.run.smithi159.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 71 kB/s | 42 kB 00:00 2024-09-17T22:54:51.965 INFO:teuthology.orchestra.run.smithi159.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 109 kB/s | 77 kB 00:00 2024-09-17T22:54:51.990 INFO:teuthology.orchestra.run.smithi159.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 75 kB/s | 51 kB 00:00 2024-09-17T22:54:52.057 INFO:teuthology.orchestra.run.smithi159.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 654 kB/s | 43 kB 00:00 2024-09-17T22:54:52.082 INFO:teuthology.orchestra.run.smithi159.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-09-17T22:54:52.106 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:54:52.108 INFO:teuthology.orchestra.run.smithi159.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:54:52.130 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:52.150 INFO:teuthology.orchestra.run.smithi159.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-17T22:54:52.184 INFO:teuthology.orchestra.run.smithi159.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-09-17T22:54:52.209 INFO:teuthology.orchestra.run.smithi159.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 312 kB/s | 18 kB 00:00 2024-09-17T22:54:52.234 INFO:teuthology.orchestra.run.smithi159.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-17T22:54:52.259 INFO:teuthology.orchestra.run.smithi159.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-17T22:54:52.285 INFO:teuthology.orchestra.run.smithi159.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-17T22:54:52.310 INFO:teuthology.orchestra.run.smithi159.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-17T22:54:52.335 INFO:teuthology.orchestra.run.smithi159.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-17T22:54:52.357 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:54:52.372 INFO:teuthology.orchestra.run.smithi159.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 907 kB/s | 79 kB 00:00 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:54:52.381 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:52.397 INFO:teuthology.orchestra.run.smithi159.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 266 kB/s | 16 kB 00:00 2024-09-17T22:54:52.423 INFO:teuthology.orchestra.run.smithi159.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T22:54:52.446 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:54:52.448 INFO:teuthology.orchestra.run.smithi159.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-17T22:54:52.473 INFO:teuthology.orchestra.run.smithi159.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-17T22:54:52.507 INFO:teuthology.orchestra.run.smithi159.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 600 kB/s | 20 kB 00:00 2024-09-17T22:54:52.532 INFO:teuthology.orchestra.run.smithi159.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-17T22:54:52.574 INFO:teuthology.orchestra.run.smithi159.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-17T22:54:52.579 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:52.579 INFO:teuthology.orchestra.run.smithi159.stdout:Total 7.5 MB/s | 30 MB 00:04 2024-09-17T22:54:52.734 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:54:52.775 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:54:52.775 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:54:53.265 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:54:53.265 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:54:54.284 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T22:54:54.285 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T22:54:54.286 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T22:54:54.287 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T22:54:54.287 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T22:54:54.287 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T22:54:54.288 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T22:54:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:54:54.736 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T22:54:54.772 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T22:54:54.804 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T22:54:54.877 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T22:54:54.920 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T22:54:54.953 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T22:54:55.044 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T22:54:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T22:54:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T22:54:55.223 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:54:55.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T22:54:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T22:54:55.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:55.232 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:54:55.253 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T22:54:55.288 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T22:54:55.321 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T22:54:55.362 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T22:54:55.405 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T22:54:55.440 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T22:54:55.488 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T22:54:55.520 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T22:54:55.593 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T22:54:55.600 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-base 2024-09-17T22:54:55.733 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T22:54:55.820 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T22:54:55.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T22:54:55.919 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T22:54:55.986 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T22:54:56.042 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T22:54:56.187 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:54:56.234 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T22:54:56.276 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:54:56.318 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:56.319 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:54:56.319 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:54:56.380 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install cephadm 2024-09-17T22:54:56.442 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T22:54:56.519 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T22:54:56.609 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T22:54:56.653 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T22:54:56.774 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T22:54:56.938 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T22:54:56.956 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:54:56.968 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:54:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:54:56.995 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:56.996 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T22:54:56.996 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:54:56.996 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:54:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:57.090 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:57.090 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 765 k 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 771 k 2024-09-17T22:54:57.091 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:54:57.353 INFO:teuthology.orchestra.run.smithi171.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-17T22:54:57.354 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:57.354 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-17T22:54:57.354 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:54:57.359 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:54:57.359 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:54:57.365 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:54:57.365 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:54:57.439 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:54:57.906 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:54:57.982 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:54:58.334 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:54:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:54:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:54:58.480 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:58.480 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T22:54:58.481 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:54:58.481 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:54:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:54:58.576 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:58.576 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:54:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:54:58.577 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:58.577 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:54:58.825 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T22:54:58.920 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:54:58.944 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:59.180 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:54:59.202 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:54:59.202 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:54:59.203 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T22:54:59.203 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:54:59.203 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:54:59.203 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:54:59.407 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:54:59.541 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:54:59.542 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T22:54:59.543 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:59.543 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:54:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T22:54:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 145 k 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 435 k 2024-09-17T22:54:59.544 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:54:59.773 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 632 kB/s | 145 kB 00:00 2024-09-17T22:54:59.773 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:54:59.774 INFO:teuthology.orchestra.run.smithi171.stdout:Total 627 kB/s | 145 kB 00:00 2024-09-17T22:54:59.774 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:54:59.782 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:54:59.782 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:54:59.838 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:54:59.839 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:54:59.985 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:00.074 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:00.100 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:00.100 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:55:00.100 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T22:55:00.100 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:00.856 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:55:00.856 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T22:55:00.857 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T22:55:00.858 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T22:55:00.859 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:01.053 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:01.332 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T22:55:01.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T22:55:01.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:01.809 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:01.926 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:02.016 INFO:teuthology.orchestra.run.smithi171.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:02.058 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:02.059 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:02.060 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:02.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T22:55:02.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-base 2024-09-17T22:55:02.709 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:02.749 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:02.841 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:02.842 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:02.842 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:02.842 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:02.842 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T22:55:02.843 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T22:55:02.844 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T22:55:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:02.845 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:02.845 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:02.845 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-17T22:55:02.845 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:02.846 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 8.7 M 2024-09-17T22:55:02.846 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 104 M 2024-09-17T22:55:02.846 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:02.884 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:02.885 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:02.885 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:02.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install cephadm 2024-09-17T22:55:03.527 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:03.658 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T22:55:03.659 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 765 k 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 771 k 2024-09-17T22:55:03.660 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:03.880 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 203 kB/s | 32 kB 00:00 2024-09-17T22:55:03.905 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 90 kB/s | 16 kB 00:00 2024-09-17T22:55:03.927 INFO:teuthology.orchestra.run.smithi159.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-17T22:55:03.928 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:03.928 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-17T22:55:03.928 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:03.933 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:03.934 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:03.939 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:03.940 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:04.013 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:04.030 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-17T22:55:04.097 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-17T22:55:04.173 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 7.9 MB/s | 3.6 MB 00:00 2024-09-17T22:55:04.223 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.3 MB/s | 548 kB 00:00 2024-09-17T22:55:04.249 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 265 kB/s | 19 kB 00:00 2024-09-17T22:55:04.299 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-17T22:55:04.333 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 923 kB/s | 31 kB 00:00 2024-09-17T22:55:04.392 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-17T22:55:04.458 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.0 MB/s | 863 kB 00:00 2024-09-17T22:55:04.472 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:04.542 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.4 MB/s | 2.0 MB 00:00 2024-09-17T22:55:04.546 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:04.546 INFO:teuthology.orchestra.run.smithi171.stdout:Total 5.1 MB/s | 8.7 MB 00:01 2024-09-17T22:55:04.564 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:04.596 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:04.613 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:04.614 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:04.821 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:04.822 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:04.958 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:05.209 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:05.366 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:05.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T22:55:05.443 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T22:55:05.487 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T22:55:05.533 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T22:55:05.564 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T22:55:05.663 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T22:55:05.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T22:55:05.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T22:55:05.946 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T22:55:05.974 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:55:05.981 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T22:55:06.117 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T22:55:06.118 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 145 k 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 435 k 2024-09-17T22:55:06.119 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:06.333 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 679 kB/s | 145 kB 00:00 2024-09-17T22:55:06.333 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:06.336 INFO:teuthology.orchestra.run.smithi159.stdout:Total 675 kB/s | 145 kB 00:00 2024-09-17T22:55:06.336 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:06.341 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:06.342 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:06.398 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:06.399 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:06.553 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:06.609 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:06.633 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:06.634 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:55:06.634 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T22:55:06.634 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:07.068 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T22:55:07.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:55:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:55:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:07.530 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:07.530 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:07.782 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr 2024-09-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T22:55:08.576 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:09.088 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi159.stdout:Install 12 Packages 2024-09-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 8.7 M 2024-09-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 104 M 2024-09-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi159.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 215 kB/s | 32 kB 00:00 2024-09-17T22:55:09.810 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:09.811 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T22:55:09.812 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout:Install 12 Packages 2024-09-17T22:55:09.813 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:09.814 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 38 M 2024-09-17T22:55:09.814 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 212 M 2024-09-17T22:55:09.814 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:09.814 INFO:teuthology.orchestra.run.smithi159.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 100 kB/s | 16 kB 00:00 2024-09-17T22:55:09.948 INFO:teuthology.orchestra.run.smithi159.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-17T22:55:10.006 INFO:teuthology.orchestra.run.smithi159.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 10 MB/s | 3.6 MB 00:00 2024-09-17T22:55:10.030 INFO:teuthology.orchestra.run.smithi171.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-17T22:55:10.048 INFO:teuthology.orchestra.run.smithi159.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-17T22:55:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-17T22:55:10.139 INFO:teuthology.orchestra.run.smithi171.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-17T22:55:10.148 INFO:teuthology.orchestra.run.smithi159.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.8 MB/s | 548 kB 00:00 2024-09-17T22:55:10.199 INFO:teuthology.orchestra.run.smithi159.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-09-17T22:55:10.224 INFO:teuthology.orchestra.run.smithi159.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-09-17T22:55:10.266 INFO:teuthology.orchestra.run.smithi159.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.3 MB/s | 188 kB 00:00 2024-09-17T22:55:10.291 INFO:teuthology.orchestra.run.smithi159.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 334 kB/s | 31 kB 00:00 2024-09-17T22:55:10.315 INFO:teuthology.orchestra.run.smithi171.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T22:55:10.333 INFO:teuthology.orchestra.run.smithi159.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.7 MB/s | 2.0 MB 00:00 2024-09-17T22:55:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:Total 8.3 MB/s | 8.7 MB 00:01 2024-09-17T22:55:10.340 INFO:teuthology.orchestra.run.smithi171.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 106 kB/s | 33 kB 00:00 2024-09-17T22:55:10.385 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:10.402 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:10.402 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:10.540 INFO:teuthology.orchestra.run.smithi171.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-17T22:55:10.599 INFO:teuthology.orchestra.run.smithi171.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-17T22:55:10.609 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:10.609 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:11.067 INFO:teuthology.orchestra.run.smithi171.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-17T22:55:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:11.226 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T22:55:11.261 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T22:55:11.283 INFO:teuthology.orchestra.run.smithi171.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-17T22:55:11.307 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T22:55:11.366 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T22:55:11.400 INFO:teuthology.orchestra.run.smithi171.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-17T22:55:11.484 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T22:55:11.592 INFO:teuthology.orchestra.run.smithi171.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.7 MB/s | 4.9 MB 00:01 2024-09-17T22:55:11.615 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T22:55:11.708 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T22:55:11.760 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T22:55:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:55:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T22:55:11.959 INFO:teuthology.orchestra.run.smithi171.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.7 MB/s | 5.1 MB 00:00 2024-09-17T22:55:12.877 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T22:55:12.912 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T22:55:14.253 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T22:55:14.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T22:55:14.570 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T22:55:14.570 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:14.570 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:14.570 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T22:55:14.571 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:14.572 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:14.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T22:55:15.407 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:06 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:15.541 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T22:55:15.542 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout:Install 12 Packages 2024-09-17T22:55:15.543 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:15.544 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 38 M 2024-09-17T22:55:15.544 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 212 M 2024-09-17T22:55:15.544 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:15.799 INFO:teuthology.orchestra.run.smithi159.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-17T22:55:15.841 INFO:teuthology.orchestra.run.smithi159.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-17T22:55:16.125 INFO:teuthology.orchestra.run.smithi159.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T22:55:16.409 INFO:teuthology.orchestra.run.smithi159.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-09-17T22:55:16.617 INFO:teuthology.orchestra.run.smithi159.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-17T22:55:16.976 INFO:teuthology.orchestra.run.smithi159.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-17T22:55:17.043 INFO:teuthology.orchestra.run.smithi159.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-17T22:55:17.718 INFO:teuthology.orchestra.run.smithi159.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.5 MB/s | 4.9 MB 00:01 2024-09-17T22:55:17.810 INFO:teuthology.orchestra.run.smithi159.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 294 kB/s | 245 kB 00:00 2024-09-17T22:55:18.268 INFO:teuthology.orchestra.run.smithi159.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-17T22:55:18.586 INFO:teuthology.orchestra.run.smithi159.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 289 kB/s | 250 kB 00:00 2024-09-17T22:55:19.252 INFO:teuthology.orchestra.run.smithi159.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T22:55:19.254 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:19.254 INFO:teuthology.orchestra.run.smithi159.stdout:Total 10 MB/s | 38 MB 00:03 2024-09-17T22:55:19.614 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:19.641 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:19.642 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:20.072 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:20.072 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:20.861 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T22:55:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:55:21.083 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:55:21.118 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T22:55:21.302 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T22:55:21.325 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T22:55:21.394 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T22:55:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T22:55:22.172 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T22:55:23.406 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:55:23.916 INFO:teuthology.orchestra.run.smithi171.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-09-17T22:55:23.917 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:23.918 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.7 MB/s | 38 MB 00:14 2024-09-17T22:55:24.280 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:24.307 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:24.307 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:24.429 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T22:55:24.458 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:55:24.746 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:24.746 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:25.583 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T22:55:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:55:25.732 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:55:25.774 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T22:55:25.960 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T22:55:25.990 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T22:55:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T22:55:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T22:55:26.865 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T22:55:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:55:27.077 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T22:55:27.574 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:27.575 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:27.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-rook 2024-09-17T22:55:28.138 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:55:28.542 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:28.676 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T22:55:28.678 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout:Install 13 Packages 2024-09-17T22:55:28.679 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:28.680 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 2.2 M 2024-09-17T22:55:28.680 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 26 M 2024-09-17T22:55:28.680 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:29.068 INFO:teuthology.orchestra.run.smithi159.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 190 kB/s | 19 kB 00:00 2024-09-17T22:55:29.093 INFO:teuthology.orchestra.run.smithi159.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 26 kB 00:00 2024-09-17T22:55:29.206 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T22:55:29.210 INFO:teuthology.orchestra.run.smithi159.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-17T22:55:29.235 INFO:teuthology.orchestra.run.smithi159.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-17T22:55:29.238 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:55:29.261 INFO:teuthology.orchestra.run.smithi159.stdout:(5/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 168 kB/s | 49 kB 00:00 2024-09-17T22:55:29.286 INFO:teuthology.orchestra.run.smithi159.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T22:55:29.311 INFO:teuthology.orchestra.run.smithi159.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-17T22:55:29.362 INFO:teuthology.orchestra.run.smithi159.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-17T22:55:29.387 INFO:teuthology.orchestra.run.smithi159.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-17T22:55:29.412 INFO:teuthology.orchestra.run.smithi159.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T22:55:29.437 INFO:teuthology.orchestra.run.smithi159.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-17T22:55:29.463 INFO:teuthology.orchestra.run.smithi159.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-17T22:55:29.496 INFO:teuthology.orchestra.run.smithi159.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-17T22:55:29.499 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:29.499 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-17T22:55:29.564 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:29.579 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:29.579 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:29.670 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:29.670 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:30.344 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T22:55:30.398 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T22:55:30.461 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T22:55:30.498 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T22:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T22:55:30.750 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T22:55:30.859 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T22:55:30.924 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T22:55:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T22:55:31.689 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T22:55:31.714 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T22:55:31.762 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T22:55:31.784 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:55:32.051 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:55:32.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T22:55:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T22:55:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:32.472 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:32.472 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:32.473 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:32.480 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T22:55:32.481 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T22:55:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T22:55:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:32.802 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:32.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-rook 2024-09-17T22:55:33.069 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T22:55:33.504 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:33.636 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T22:55:33.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:33.638 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:Install 13 Packages 2024-09-17T22:55:33.639 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 2.2 M 2024-09-17T22:55:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 26 M 2024-09-17T22:55:33.639 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:33.652 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:24 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:33.786 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:33.786 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:33.787 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout:Install 6 Packages 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 7.0 M 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 33 M 2024-09-17T22:55:33.788 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:33.963 INFO:teuthology.orchestra.run.smithi171.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 278 kB/s | 26 kB 00:00 2024-09-17T22:55:33.988 INFO:teuthology.orchestra.run.smithi171.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 163 kB/s | 19 kB 00:00 2024-09-17T22:55:34.071 INFO:teuthology.orchestra.run.smithi171.stdout:(3/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 245 kB/s | 49 kB 00:00 2024-09-17T22:55:34.130 INFO:teuthology.orchestra.run.smithi171.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-17T22:55:34.172 INFO:teuthology.orchestra.run.smithi171.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 863 kB/s | 159 kB 00:00 2024-09-17T22:55:34.189 INFO:teuthology.orchestra.run.smithi171.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-17T22:55:34.223 INFO:teuthology.orchestra.run.smithi171.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 638 kB/s | 32 kB 00:00 2024-09-17T22:55:34.240 INFO:teuthology.orchestra.run.smithi171.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-17T22:55:34.273 INFO:teuthology.orchestra.run.smithi171.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-17T22:55:34.299 INFO:teuthology.orchestra.run.smithi171.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-17T22:55:34.324 INFO:teuthology.orchestra.run.smithi171.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T22:55:34.374 INFO:teuthology.orchestra.run.smithi171.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-17T22:55:34.408 INFO:teuthology.orchestra.run.smithi171.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-09-17T22:55:34.411 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:34.411 INFO:teuthology.orchestra.run.smithi171.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-17T22:55:34.477 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:34.492 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:34.492 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:34.583 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:34.585 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:34.631 INFO:teuthology.orchestra.run.smithi159.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 674 kB/s | 146 kB 00:00 2024-09-17T22:55:34.715 INFO:teuthology.orchestra.run.smithi159.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-17T22:55:34.740 INFO:teuthology.orchestra.run.smithi159.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-17T22:55:34.765 INFO:teuthology.orchestra.run.smithi159.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-17T22:55:35.106 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:35.141 INFO:teuthology.orchestra.run.smithi159.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 342 kB/s | 248 kB 00:00 2024-09-17T22:55:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T22:55:35.399 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T22:55:35.436 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T22:55:35.478 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T22:55:35.581 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T22:55:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T22:55:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T22:55:36.467 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T22:55:36.513 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T22:55:36.547 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T22:55:36.595 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T22:55:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T22:55:37.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T22:55:37.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T22:55:37.754 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T22:55:37.754 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T22:55:37.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T22:55:37.756 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:37.756 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:38.031 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T22:55:38.074 INFO:teuthology.orchestra.run.smithi159.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.6 MB/s | 6.0 MB 00:03 2024-09-17T22:55:38.076 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:38.077 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.6 MB/s | 7.0 MB 00:04 2024-09-17T22:55:38.149 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:38.161 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:38.161 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:38.260 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:38.261 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:38.567 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:38.610 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T22:55:38.743 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:38.744 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:38.744 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:38.744 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:38.744 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout:Install 6 Packages 2024-09-17T22:55:38.745 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:38.746 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 7.0 M 2024-09-17T22:55:38.746 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 33 M 2024-09-17T22:55:38.746 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T22:55:39.318 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T22:55:39.368 INFO:teuthology.orchestra.run.smithi171.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-17T22:55:39.393 INFO:teuthology.orchestra.run.smithi171.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 663 kB/s | 146 kB 00:00 2024-09-17T22:55:39.403 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T22:55:39.460 INFO:teuthology.orchestra.run.smithi171.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 864 kB/s | 58 kB 00:00 2024-09-17T22:55:39.467 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T22:55:39.488 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:55:39.493 INFO:teuthology.orchestra.run.smithi171.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-17T22:55:39.536 INFO:teuthology.orchestra.run.smithi171.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.2 MB/s | 548 kB 00:00 2024-09-17T22:55:40.236 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:55:40.237 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T22:55:40.237 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T22:55:40.237 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T22:55:40.237 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T22:55:40.237 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T22:55:40.512 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T22:55:40.512 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T22:55:40.513 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T22:55:40.514 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T22:55:40.514 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:40.514 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:40.619 INFO:teuthology.orchestra.run.smithi171.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-17T22:55:40.657 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:40.657 INFO:teuthology.orchestra.run.smithi171.stdout:Total 3.7 MB/s | 7.0 MB 00:01 2024-09-17T22:55:40.699 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:40.711 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:40.711 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:40.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-fuse 2024-09-17T22:55:40.814 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:40.814 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:41.115 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:41.284 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T22:55:41.323 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:41.347 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T22:55:41.455 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:41.455 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:41.455 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:41.455 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:41.455 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout:Install 2 Packages 2024-09-17T22:55:41.456 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:41.457 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 977 k 2024-09-17T22:55:41.457 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 2.9 M 2024-09-17T22:55:41.457 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:41.851 INFO:teuthology.orchestra.run.smithi159.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 692 kB/s | 80 kB 00:00 2024-09-17T22:55:41.870 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T22:55:41.958 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T22:55:42.022 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T22:55:42.026 INFO:teuthology.orchestra.run.smithi159.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.0 MB/s | 897 kB 00:00 2024-09-17T22:55:42.026 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:42.027 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.7 MB/s | 977 kB 00:00 2024-09-17T22:55:42.038 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:42.052 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:55:42.059 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:42.060 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:42.140 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:42.141 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:42.329 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:42.471 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T22:55:42.535 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:55:42.828 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:55:42.829 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T22:55:42.829 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T22:55:42.829 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T22:55:42.829 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T22:55:42.829 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T22:55:43.093 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:43.094 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:43.254 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:55:43.254 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T22:55:43.379 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-fuse 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:43.516 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:43.838 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-volume 2024-09-17T22:55:43.963 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:44.095 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-17T22:55:44.096 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout:Install 2 Packages 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:44.097 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 977 k 2024-09-17T22:55:44.098 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 2.9 M 2024-09-17T22:55:44.098 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:44.383 INFO:teuthology.orchestra.run.smithi171.stdout:(1/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.4 MB/s | 897 kB 00:00 2024-09-17T22:55:44.430 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:44.567 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:44.567 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:44.568 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:44.569 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 285 k 2024-09-17T22:55:44.569 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 1.3 M 2024-09-17T22:55:44.569 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:44.797 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T22:55:44.797 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:44.798 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T22:55:44.798 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:44.806 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:44.806 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:44.830 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:44.831 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:44.850 INFO:teuthology.orchestra.run.smithi171.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 110 kB/s | 80 kB 00:00 2024-09-17T22:55:44.851 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:44.851 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.3 MB/s | 977 kB 00:00 2024-09-17T22:55:44.864 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:44.887 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:44.887 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:44.969 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:44.969 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:45.099 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:45.115 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:45.116 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:55:45.116 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T22:55:45.116 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:45.159 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:45.299 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T22:55:45.375 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:55:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:55:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:46.341 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:46.364 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados-devel 2024-09-17T22:55:46.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-volume 2024-09-17T22:55:46.964 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:47.101 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:47.102 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 127 k 2024-09-17T22:55:47.103 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 456 k 2024-09-17T22:55:47.103 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:47.209 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:47.294 INFO:teuthology.orchestra.run.smithi159.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 661 kB/s | 127 kB 00:00 2024-09-17T22:55:47.295 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:47.295 INFO:teuthology.orchestra.run.smithi159.stdout:Total 656 kB/s | 127 kB 00:00 2024-09-17T22:55:47.295 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:47.300 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:47.301 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:47.351 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:47.351 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:47.351 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 285 k 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 1.3 M 2024-09-17T22:55:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:47.364 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:47.365 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:47.521 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:47.584 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T22:55:47.584 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:47.584 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T22:55:47.585 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:47.593 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:47.593 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:47.616 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:47.617 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:47.624 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:47.835 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:47.911 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:47.928 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:55:47.928 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T22:55:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:48.086 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:48.336 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:48.623 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs2 2024-09-17T22:55:48.960 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:55:48.961 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:48.961 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:48.961 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:55:48.961 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:48.961 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:49.221 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:49.263 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados-devel 2024-09-17T22:55:49.313 INFO:teuthology.orchestra.run.smithi159.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:49.358 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:49.359 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:49.359 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:49.426 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs-devel 2024-09-17T22:55:49.855 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:49.991 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:49.992 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:49.993 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:55:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:49.993 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 127 k 2024-09-17T22:55:49.993 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 456 k 2024-09-17T22:55:49.993 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:50.009 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:50.138 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:50.138 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 32 k 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 142 k 2024-09-17T22:55:50.139 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:50.202 INFO:teuthology.orchestra.run.smithi171.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 605 kB/s | 127 kB 00:00 2024-09-17T22:55:50.203 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:50.203 INFO:teuthology.orchestra.run.smithi171.stdout:Total 600 kB/s | 127 kB 00:00 2024-09-17T22:55:50.204 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:50.212 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:50.212 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:50.274 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:50.274 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:50.297 INFO:teuthology.orchestra.run.smithi159.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 201 kB/s | 32 kB 00:00 2024-09-17T22:55:50.298 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:50.298 INFO:teuthology.orchestra.run.smithi159.stdout:Total 199 kB/s | 32 kB 00:00 2024-09-17T22:55:50.298 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:50.302 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:50.302 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:50.323 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:50.324 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:50.416 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:50.422 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:50.512 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:50.882 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:50.989 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:51.095 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:51.242 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:51.242 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:51.243 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:51.243 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:51.243 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:51.243 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:51.285 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados2 2024-09-17T22:55:51.498 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs2 2024-09-17T22:55:51.858 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:51.948 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:51.991 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:51.992 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:51.992 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:52.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librbd1 2024-09-17T22:55:52.089 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:52.182 INFO:teuthology.orchestra.run.smithi171.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:52.228 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:52.228 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:52.229 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:52.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install libcephfs-devel 2024-09-17T22:55:52.644 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:52.734 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:52.777 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:52.778 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:52.778 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:52.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rados 2024-09-17T22:55:52.886 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:53.019 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:53.020 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:53.020 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T22:55:53.020 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:53.020 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 32 k 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 142 k 2024-09-17T22:55:53.021 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:53.294 INFO:teuthology.orchestra.run.smithi171.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 116 kB/s | 32 kB 00:00 2024-09-17T22:55:53.294 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:53.295 INFO:teuthology.orchestra.run.smithi171.stdout:Total 116 kB/s | 32 kB 00:00 2024-09-17T22:55:53.295 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:55:53.298 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:55:53.298 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:55:53.319 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:55:53.319 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:55:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:55:53.425 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:53.515 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:53.558 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:53.559 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:53.559 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:53.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rgw 2024-09-17T22:55:53.914 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:54.141 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:54.232 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:54.327 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:54.335 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librados2 2024-09-17T22:55:54.371 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:54.373 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:54.373 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:54.448 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-cephfs 2024-09-17T22:55:54.914 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:55.005 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:55.048 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:55.049 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:55.049 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:55.062 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:55.124 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install librbd1 2024-09-17T22:55:55.154 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:55.199 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:55.200 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:55.200 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:55.269 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rbd 2024-09-17T22:55:55.702 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:55.791 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:55.835 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:55.836 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:55.836 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:55.865 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:55.911 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rados 2024-09-17T22:55:55.955 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:55.998 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:55.999 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2024-09-17T22:55:55.999 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:56.060 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-fuse 2024-09-17T22:55:56.499 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:56.590 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:56.634 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:56.635 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:56.635 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:56.635 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:56.711 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rgw 2024-09-17T22:55:56.768 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:56.768 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 85 k 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 231 k 2024-09-17T22:55:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:57.018 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 344 kB/s | 85 kB 00:00 2024-09-17T22:55:57.018 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:57.018 INFO:teuthology.orchestra.run.smithi159.stdout:Total 342 kB/s | 85 kB 00:00 2024-09-17T22:55:57.019 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:57.025 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:57.026 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:57.088 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:57.089 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:57.250 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:57.295 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:57.348 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:57.384 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:57.427 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:57.428 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:57.428 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:57.500 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-cephfs 2024-09-17T22:55:57.733 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:55:58.088 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:58.176 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:58.219 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:58.220 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:58.220 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:58.243 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-mirror 2024-09-17T22:55:58.289 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install python3-rbd 2024-09-17T22:55:58.846 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:55:58.880 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:58.970 INFO:teuthology.orchestra.run.smithi171.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:55:58.983 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:55:58.984 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:58.984 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2024-09-17T22:55:58.984 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:58.984 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:55:58.985 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:58.986 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 3.1 M 2024-09-17T22:55:58.986 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 13 M 2024-09-17T22:55:58.986 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:55:59.013 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:59.014 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-17T22:55:59.014 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:55:59.088 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-fuse 2024-09-17T22:55:59.321 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-17T22:55:59.322 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:59.322 INFO:teuthology.orchestra.run.smithi159.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-17T22:55:59.323 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:55:59.331 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:55:59.332 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:55:59.412 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:55:59.413 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:55:59.663 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:55 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:55:59.796 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:55:59.797 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:55:59.798 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 85 k 2024-09-17T22:55:59.798 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 231 k 2024-09-17T22:55:59.798 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:55:59.846 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:55:59.914 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:59.939 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:55:59.939 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:55:59.939 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T22:55:59.939 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:55:59.939 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:55:59.940 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:55:59.998 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 426 kB/s | 85 kB 00:00 2024-09-17T22:55:59.998 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:55:59.999 INFO:teuthology.orchestra.run.smithi171.stdout:Total 424 kB/s | 85 kB 00:00 2024-09-17T22:55:59.999 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:56:00.006 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:56:00.006 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:56:00.069 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:56:00.069 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:56:00.220 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:56:00.345 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:00.739 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:00.985 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:01.056 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:56:01.254 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-mirror 2024-09-17T22:56:01.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-nbd 2024-09-17T22:56:01.863 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:01:57 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:56:01.917 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 10:54:09 PM UTC. 2024-09-17T22:56:02.003 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repo Size 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:02.004 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 3.1 M 2024-09-17T22:56:02.005 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 13 M 2024-09-17T22:56:02.005 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:56:02.051 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-17T22:56:02.051 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:02.052 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 170 k 2024-09-17T22:56:02.053 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 494 k 2024-09-17T22:56:02.053 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-17T22:56:02.267 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 797 kB/s | 170 kB 00:00 2024-09-17T22:56:02.267 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:56:02.267 INFO:teuthology.orchestra.run.smithi159.stdout:Total 791 kB/s | 170 kB 00:00 2024-09-17T22:56:02.268 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-17T22:56:02.275 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-17T22:56:02.275 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-17T22:56:02.338 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-17T22:56:02.338 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-17T22:56:02.339 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.3 MB/s | 3.1 MB 00:00 2024-09-17T22:56:02.339 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:56:02.340 INFO:teuthology.orchestra.run.smithi171.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-17T22:56:02.340 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:56:02.349 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:56:02.349 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:56:02.432 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:56:02.432 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:56:02.493 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-17T22:56:02.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:56:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:56:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:03.032 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:03.286 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-17T22:56:03.537 DEBUG:teuthology.parallel:result is None 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:04.013 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:56:04.259 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install rbd-nbd 2024-09-17T22:56:04.849 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:02:00 ago on Tue 17 Sep 2024 10:54:04 PM UTC. 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T22:56:04.985 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:Install 1 Package 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 170 k 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 494 k 2024-09-17T22:56:04.986 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-17T22:56:05.200 INFO:teuthology.orchestra.run.smithi171.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 793 kB/s | 170 kB 00:00 2024-09-17T22:56:05.201 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:56:05.201 INFO:teuthology.orchestra.run.smithi171.stdout:Total 787 kB/s | 170 kB 00:00 2024-09-17T22:56:05.201 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-17T22:56:05.208 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-17T22:56:05.209 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-17T22:56:05.272 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-17T22:56:05.272 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-17T22:56:05.422 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-17T22:56:05.514 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:05.973 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:06.241 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-17T22:56:06.476 DEBUG:teuthology.parallel:result is None 2024-09-17T22:56:06.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:56:06.627 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T22:56:06.653 INFO:teuthology.orchestra.run.smithi159.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T22:56:06.653 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T22:56:06.654 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T22:56:06.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:56:06.800 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T22:56:06.823 INFO:teuthology.orchestra.run.smithi171.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T22:56:06.823 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T22:56:06.823 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T22:56:06.826 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T22:56:06.826 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:06.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T22:56:06.857 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:06.857 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T22:56:06.888 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T22:56:06.888 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:06.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T22:56:06.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T22:56:06.996 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:06.996 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T22:56:07.021 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T22:56:07.086 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T22:56:07.086 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:07.086 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T22:56:07.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T22:56:07.181 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:07.181 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T22:56:07.205 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T22:56:07.271 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T22:56:07.271 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:07.271 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T22:56:07.298 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T22:56:07.366 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:07.366 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T22:56:07.391 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T22:56:07.456 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T22:56:07.530 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T22:56:07.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /var/log/ceph 2024-09-17T22:56:07.533 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-09-17T22:56:07.559 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T22:56:07.559 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T22:56:07.601 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T22:56:07.626 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T22:56:07.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T22:56:07.668 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T22:56:07.703 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T22:56:07.703 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T22:56:07.703 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909554', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi159.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBP3k32Kst8tQSHniPqTT7yleeKRhKtdF1o1YlGImb9Z0J8kHuwpVSfj0vCjhajpLd6ySYyIt0gQkIEt6m9hApw=', 'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGHc2LG74uPPb3WAok5lbxqcDxRG3Oa26xCeBMmquyrnFuqq9MjLOGNXm6/UbWFZC3vP+iBh1dJcs3odAcURWrc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T22:56:07.704 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T22:56:07.735 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T22:56:07.751 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T22:56:07.791 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T22:56:07.815 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:07.816 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T22:56:07.870 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T22:56:07.870 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-09-17T22:56:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T22:56:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:07.926 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T22:56:07.926 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:07.927 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:07.927 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-17 22:56:00.188365990 +0000 2024-09-17T22:56:07.927 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-17 22:53:25.448416199 +0000 2024-09-17T22:56:07.927 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-17 22:53:25.448416199 +0000 2024-09-17T22:56:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-17 22:53:25.448416199 +0000 2024-09-17T22:56:07.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T22:56:07.990 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-17T22:56:07.990 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-17T22:56:07.990 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000211319 s, 2.4 MB/s 2024-09-17T22:56:07.991 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T22:56:08.048 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:08.104 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-17 22:56:00.188365990 +0000 2024-09-17T22:56:08.105 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-17 22:53:25.774410199 +0000 2024-09-17T22:56:08.105 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-17 22:53:25.774410199 +0000 2024-09-17T22:56:08.105 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-17 22:53:25.774410199 +0000 2024-09-17T22:56:08.105 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T22:56:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-17T22:56:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-17T22:56:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000193832 s, 2.6 MB/s 2024-09-17T22:56:08.170 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T22:56:08.225 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-09-17T22:56:08.280 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T22:56:08.280 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-17 22:56:00.189365971 +0000 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-17 22:53:26.109404034 +0000 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-17 22:53:26.109404034 +0000 2024-09-17T22:56:08.281 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-17 22:53:26.109404034 +0000 2024-09-17T22:56:08.281 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T22:56:08.343 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-17T22:56:08.344 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-17T22:56:08.344 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000230853 s, 2.2 MB/s 2024-09-17T22:56:08.345 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T22:56:08.401 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-17 22:56:00.189365971 +0000 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-17 22:53:26.460397574 +0000 2024-09-17T22:56:08.457 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-17 22:53:26.460397574 +0000 2024-09-17T22:56:08.458 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-17 22:53:26.460397574 +0000 2024-09-17T22:56:08.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T22:56:08.520 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-17T22:56:08.520 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-17T22:56:08.520 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000194439 s, 2.6 MB/s 2024-09-17T22:56:08.521 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T22:56:08.577 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:08.578 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:08.578 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T22:56:08.595 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T22:56:08.595 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-17T22:56:08.651 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 22:56:03.209843934 +0000 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 22:53:26.392067378 +0000 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 22:53:26.392067378 +0000 2024-09-17T22:56:08.652 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 22:53:26.392067378 +0000 2024-09-17T22:56:08.652 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T22:56:08.717 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T22:56:08.717 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T22:56:08.717 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000207342 s, 2.5 MB/s 2024-09-17T22:56:08.718 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T22:56:08.775 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:08.832 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 22:56:03.210843914 +0000 2024-09-17T22:56:08.833 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 22:53:26.728060817 +0000 2024-09-17T22:56:08.833 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 22:53:26.728060817 +0000 2024-09-17T22:56:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 22:53:26.728060817 +0000 2024-09-17T22:56:08.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T22:56:08.896 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T22:56:08.896 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T22:56:08.896 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00021942 s, 2.3 MB/s 2024-09-17T22:56:08.897 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T22:56:08.953 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-17T22:56:09.008 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 22:56:03.210843914 +0000 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 22:53:27.069054157 +0000 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 22:53:27.069054157 +0000 2024-09-17T22:56:09.009 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 22:53:27.069054157 +0000 2024-09-17T22:56:09.009 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T22:56:09.072 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T22:56:09.072 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T22:56:09.072 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000191763 s, 2.7 MB/s 2024-09-17T22:56:09.073 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T22:56:09.129 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-17T22:56:09.184 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-17 22:56:03.210843914 +0000 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-17 22:53:27.394047811 +0000 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-17 22:53:27.394047811 +0000 2024-09-17T22:56:09.185 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-17 22:53:27.394047811 +0000 2024-09-17T22:56:09.185 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T22:56:09.249 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-17T22:56:09.249 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-17T22:56:09.249 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000243324 s, 2.1 MB/s 2024-09-17T22:56:09.250 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T22:56:09.306 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:09.306 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T22:56:09.306 INFO:tasks.ceph:Generating config... 2024-09-17T22:56:09.307 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-17T22:56:09.308 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-17T22:56:09.308 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-17T22:56:09.308 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T22:56:09.308 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T22:56:09.308 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T22:56:09.308 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T22:56:09.308 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T22:56:09.308 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T22:56:09.308 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T22:56:09.308 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T22:56:09.308 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-17T22:56:09.308 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-17T22:56:09.308 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T22:56:09.309 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-17T22:56:09.310 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T22:56:09.310 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T22:56:09.363 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T22:56:09.366 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T22:56:09.411 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:56:09.437 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.159'), ('mon.c', '[v2:172.21.15.159:3301,v1:172.21.15.159:6790]'), ('mon.b', '172.21.15.171')] 2024-09-17T22:56:09.438 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.159,[v2:172.21.15.159:3301,v1:172.21.15.159:6790],172.21.15.171', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T22:56:09.438 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:09.438 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T22:56:09.493 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.159 --addv c '[v2:172.21.15.159:3301,v1:172.21.15.159:6790]' --add b 172.21.15.171 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:09.580 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: generated fsid 33081d94-d15e-4bd4-ba4c-30022eb1be7a 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:setting min_mon_release = quincy 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:epoch 0 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:fsid 33081d94-d15e-4bd4-ba4c-30022eb1be7a 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:last_changed 2024-09-17T22:56:09.579238+0000 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:created 2024-09-17T22:56:09.579238+0000 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:min_mon_release 17 (quincy) 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:election_strategy: 1 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-17T22:56:09.581 INFO:teuthology.orchestra.run.smithi159.stdout:2: [v2:172.21.15.159:3301/0,v1:172.21.15.159:6790/0] mon.c 2024-09-17T22:56:09.582 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T22:56:09.583 DEBUG:teuthology.orchestra.run.smithi159:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T22:56:09.598 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 33081d94-d15e-4bd4-ba4c-30022eb1be7a... 2024-09-17T22:56:09.599 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T22:56:09.641 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T22:56:09.681 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-09-17T22:56:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-09-17T22:56:09.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.684 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,[v2:172.21.15.159:3301,v1:172.21.15.159:6790],172.21.15.171" 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon client directed command retry = 5 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on pool no app = False 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: ms inject socket failures = 5000 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 33081d94-d15e-4bd4-ba4c-30022eb1be7a 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-09-17T22:56:09.685 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-09-17T22:56:09.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.687 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: bluefs allocator = hybrid 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = hybrid 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-09-17T22:56:09.688 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: osd blocked scrub grace period = 3600 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: osd max markdown count = 1000 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: osd shutdown pgref assert = True 2024-09-17T22:56:09.689 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-09-17T22:56:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T22:56:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: rbd qos iops burst = 100 2024-09-17T22:56:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: rbd qos iops limit = 50 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi159.stdout: rbd qos schedule tick min = 100 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-17T22:56:09.693 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.694 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-17T22:56:09.695 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-17T22:56:09.696 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.159,[v2:172.21.15.159:3301,v1:172.21.15.159:6790],172.21.15.171" 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: mon client directed command retry = 5 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: ms inject socket failures = 5000 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33081d94-d15e-4bd4-ba4c-30022eb1be7a 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-17T22:56:09.697 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.699 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-17T22:56:09.699 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug trim objects = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.700 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: bluefs allocator = hybrid 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = hybrid 2024-09-17T22:56:09.701 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: osd blocked scrub grace period = 3600 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: osd max markdown count = 1000 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-17T22:56:09.702 INFO:teuthology.orchestra.run.smithi171.stdout: osd shutdown pgref assert = True 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-17T22:56:09.703 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T22:56:09.704 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-17T22:56:09.705 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos iops burst = 100 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos iops limit = 50 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout: rbd qos schedule tick min = 100 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-17T22:56:09.706 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-17T22:56:09.708 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T22:56:09.708 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T22:56:09.802 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T22:56:09.802 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:09.802 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T22:56:09.819 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:09.819 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T22:56:09.873 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:56:09.873 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:09.874 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T22:56:09.874 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:56:09.947 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:09.948 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:10.003 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:56:10.003 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:10.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T22:56:10.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:56:10.038 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:10.038 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:10.093 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T22:56:10.094 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T22:56:10.094 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T22:56:10.145 INFO:teuthology.orchestra.run.smithi159.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T22:56:10.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T22:56:10.223 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T22:56:10.225 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T22:56:10.225 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T22:56:10.226 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T22:56:10.267 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T22:56:10.280 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T22:56:10.321 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T22:56:10.333 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T22:56:10.333 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T22:56:10.334 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T22:56:10.358 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:10.359 INFO:tasks.ceph:role: osd.0 2024-09-17T22:56:10.359 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:56:10.359 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T22:56:10.430 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:10.431 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:10.936 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-09-17T22:56:10.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-09-17T22:56:10.938 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T22:56:11.055 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T22:56:11.086 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T22:56:11.151 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:11.151 INFO:tasks.ceph:role: osd.1 2024-09-17T22:56:11.151 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:56:11.151 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:11.224 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:11.225 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:11.225 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:11.225 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:11.685 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-09-17T22:56:11.687 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-09-17T22:56:11.687 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T22:56:11.779 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T22:56:11.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T22:56:11.872 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:11.872 INFO:tasks.ceph:role: osd.2 2024-09-17T22:56:11.872 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:56:11.872 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T22:56:11.944 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:11.944 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:11.944 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:11.944 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:11.945 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:12.433 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-09-17T22:56:12.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-09-17T22:56:12.435 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T22:56:12.577 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T22:56:12.606 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T22:56:12.671 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:12.671 INFO:tasks.ceph:role: osd.3 2024-09-17T22:56:12.671 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:56:12.671 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T22:56:12.743 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:12.744 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:13.262 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-09-17T22:56:13.264 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-09-17T22:56:13.264 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T22:56:13.389 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T22:56:13.421 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:13.515+0000 7f378f2a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T22:56:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:13.516+0000 7f378f2a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T22:56:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:13.516+0000 7f378f2a5640 -1 bdev(0x55b0dbfeb800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:13.516+0000 7f378f2a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T22:56:17.171 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T22:56:17.202 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:17.289+0000 7ffa63b23640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T22:56:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:17.290+0000 7ffa63b23640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T22:56:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:17.290+0000 7ffa63b23640 -1 bdev(0x5620ae35d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:17.290+0000 7ffa63b23640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T22:56:20.199 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T22:56:20.225 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:20.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:20.310+0000 7f4b45c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T22:56:20.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:20.310+0000 7f4b45c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T22:56:20.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:20.310+0000 7f4b45c11640 -1 bdev(0x559173633800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:20.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:20.311+0000 7f4b45c11640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T22:56:23.249 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T22:56:23.275 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:23.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:23.358+0000 7f2e495c1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T22:56:23.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:23.358+0000 7f2e495c1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T22:56:23.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:23.358+0000 7f2e495c1640 -1 bdev(0x560c6d381800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:23.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T22:56:23.358+0000 7f2e495c1640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T22:56:26.515 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T22:56:26.544 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T22:56:26.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:26.576 INFO:tasks.ceph:role: osd.4 2024-09-17T22:56:26.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:56:26.576 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T22:56:26.652 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:26.652 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:26.653 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:26.654 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:27.177 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T22:56:27.179 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T22:56:27.180 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T22:56:27.318 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T22:56:27.355 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T22:56:27.426 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:27.426 INFO:tasks.ceph:role: osd.5 2024-09-17T22:56:27.426 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:56:27.426 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T22:56:27.501 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:27.501 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:27.501 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:27.501 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:27.502 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:27.943 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T22:56:27.945 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T22:56:27.946 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T22:56:28.036 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T22:56:28.067 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T22:56:28.133 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:28.134 INFO:tasks.ceph:role: osd.6 2024-09-17T22:56:28.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:56:28.134 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:28.207 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:28.208 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:28.208 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:28.697 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T22:56:28.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T22:56:28.699 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T22:56:28.838 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T22:56:28.870 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T22:56:28.935 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:56:28.935 INFO:tasks.ceph:role: osd.7 2024-09-17T22:56:28.935 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:56:28.935 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T22:56:29.007 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:56:29.008 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:56:29.491 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-09-17T22:56:29.493 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-17T22:56:29.494 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T22:56:29.621 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T22:56:29.657 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:29.755 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:29.752+0000 7f7bdec0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T22:56:29.755 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:29.753+0000 7f7bdec0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T22:56:29.756 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:29.753+0000 7f7bdec0c640 -1 bdev(0x55d839283800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:29.756 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:29.753+0000 7f7bdec0c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T22:56:32.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T22:56:32.949 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:33.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:33.036+0000 7feca2c9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T22:56:33.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:33.037+0000 7feca2c9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T22:56:33.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:33.037+0000 7feca2c9c640 -1 bdev(0x561c0735b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:33.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:33.037+0000 7feca2c9c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T22:56:36.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T22:56:36.481 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:36.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:36.569+0000 7f34a2943640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T22:56:36.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:36.569+0000 7f34a2943640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T22:56:36.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:36.569+0000 7f34a2943640 -1 bdev(0x563fa085b800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:36.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:36.570+0000 7f34a2943640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T22:56:39.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T22:56:39.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:39.838 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:39.835+0000 7f9894545640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T22:56:39.838 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:39.835+0000 7f9894545640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T22:56:39.838 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:39.835+0000 7f9894545640 -1 bdev(0x5612fe799800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T22:56:39.839 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-17T22:56:39.835+0000 7f9894545640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T22:56:42.754 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T22:56:42.784 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T22:56:42.784 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:42.785 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T22:56:42.816 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:42.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T22:56:42.886 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:42.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T22:56:42.955 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:42.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T22:56:43.022 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:43.022 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T22:56:43.089 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.089 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T22:56:43.123 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T22:56:43.193 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T22:56:43.261 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.262 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T22:56:43.332 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.332 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T22:56:43.400 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:43.400 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T22:56:43.419 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:43.419 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T22:56:43.457 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T22:56:43.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T22:56:43.462 DEBUG:teuthology.orchestra.run.smithi171:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr.y] 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAKCepmHe6NCBAA1MBSrEEuJsNFuBm06I1oCA== 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.0] 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQANCepmCe3NHhAACv5LBgAEA1lwA8EqQu0zfA== 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.1] 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQARCepmfzdXERAAfRyl91RRU3blovqYCj3lBQ== 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.2] 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAUCepm5UCSEhAA0GfkVspUhSl309YO54xM/Q== 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.3] 2024-09-17T22:56:43.501 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAXCepm9XtvFRAA6iMr7Ju48mFi2XoE5sVT/w== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr.x] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAKCepmuDE1DRAA/mBD2YTTlqcwEjgAFayV0g== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.4] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAdCepmxvHqLBAAB01UHfhe/JiILqXqImL+7g== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.5] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAhCepmeUo8AhAAlZIM6Oeylzx/tWetWRulVQ== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.6] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAkCepmJGf+IRAAil+5k9sGXfYCfAcdQL2Vdg== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.7] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAnCepmBGvWMRAAzKWPq7k2/U9qSbUitRHFQg== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAKCepmy9DZDxAA1VdxTgeN0rH7t/TsyaRESg== 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout:[client.1] 2024-09-17T22:56:43.502 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAKCepmhhcNExAAAWrnF4ZbfhFpCS01NqKq4A== 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.y] 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAKCepmHe6NCBAA1MBSrEEuJsNFuBm06I1oCA== 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.0] 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQANCepmCe3NHhAACv5LBgAEA1lwA8EqQu0zfA== 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.1] 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQARCepmfzdXERAAfRyl91RRU3blovqYCj3lBQ== 2024-09-17T22:56:43.523 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.2] 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAUCepm5UCSEhAA0GfkVspUhSl309YO54xM/Q== 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.3] 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAXCepm9XtvFRAA6iMr7Ju48mFi2XoE5sVT/w== 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.x] 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAKCepmuDE1DRAA/mBD2YTTlqcwEjgAFayV0g== 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.4] 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAdCepmxvHqLBAAB01UHfhe/JiILqXqImL+7g== 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.5] 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAhCepmeUo8AhAAlZIM6Oeylzx/tWetWRulVQ== 2024-09-17T22:56:43.524 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.6] 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAkCepmJGf+IRAAil+5k9sGXfYCfAcdQL2Vdg== 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.7] 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAnCepmBGvWMRAAzKWPq7k2/U9qSbUitRHFQg== 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout:[client.0] 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAKCepmy9DZDxAA1VdxTgeN0rH7t/TsyaRESg== 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout:[client.1] 2024-09-17T22:56:43.525 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAKCepmhhcNExAAAWrnF4ZbfhFpCS01NqKq4A== 2024-09-17T22:56:43.526 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:56:43.546 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:56:43.620 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.727 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.730 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.784 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.835 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:56:43.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:56:43.891 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.944 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.946 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:43.998 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:44.000 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:44.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:44.053 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:56:44.104 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:56:44.108 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:56:44.161 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:56:44.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:56:44.217 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T22:56:44.217 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T22:56:44.242 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:56:44.337 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T22:56:44.362 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T22:56:44.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:56:44.523 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T22:56:44.548 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T22:56:44.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:56:44.671 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T22:56:44.695 DEBUG:teuthology.orchestra.run.smithi159:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:44.698 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:56:44.752 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T22:56:44.753 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T22:56:44.753 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T22:56:44.756 INFO:tasks.ceph.mon.a:Started 2024-09-17T22:56:44.756 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T22:56:44.756 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T22:56:44.758 INFO:tasks.ceph.mon.c:Started 2024-09-17T22:56:44.758 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T22:56:44.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T22:56:44.795 INFO:tasks.ceph.mon.b:Started 2024-09-17T22:56:44.795 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T22:56:44.795 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T22:56:44.795 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T22:56:44.798 INFO:tasks.ceph.mgr.y:Started 2024-09-17T22:56:44.798 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T22:56:44.798 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T22:56:44.800 INFO:tasks.ceph.mgr.x:Started 2024-09-17T22:56:44.800 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T22:56:44.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T22:56:50.186 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.182+0000 7f27823d6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T22:56:50.228 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.224+0000 7f27823d6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T22:56:50.368 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.364+0000 7f27823d6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T22:56:50.417 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.414+0000 7f27823d6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T22:56:50.574 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.571+0000 7f27823d6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:56:50.683 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:50.680+0000 7f27823d6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T22:56:50.981 INFO:teuthology.orchestra.run.smithi159.stderr:adjusted tunables profile to default 2024-09-17T22:56:50.993 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T22:56:50.993 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T22:56:50.994 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:50.994 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T22:56:51.014 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:51.011+0000 7f27823d6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T22:56:51.019 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:51.019 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T22:56:51.065 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:51.062+0000 7f27823d6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T22:56:51.085 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:51.085 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T22:56:51.151 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-17T22:56:51.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T22:56:51.214 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:51.211+0000 7f27823d6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T22:56:51.215 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:51.215 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T22:56:51.240 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:51.240 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T22:56:51.271 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.268+0000 7fb1be521040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T22:56:51.304 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:51.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T22:56:51.311 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.308+0000 7fb1be521040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T22:56:51.370 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-17T22:56:51.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T22:56:51.435 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 892321c3-409f-40e6-bfe1-c87f464d030f 0 2024-09-17T22:56:51.453 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.450+0000 7fb1be521040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T22:56:51.502 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.500+0000 7fb1be521040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T22:56:51.656 INFO:teuthology.orchestra.run.smithi171.stdout:0 2024-09-17T22:56:51.659 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.657+0000 7fb1be521040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:56:51.667 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new cd777f15-a6ad-4f0c-820b-ef529dd9f008 1 2024-09-17T22:56:51.769 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:51.766+0000 7fb1be521040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T22:56:51.849 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-17T22:56:51.860 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new bd68e905-6e66-4735-9de8-66a99530420f 2 2024-09-17T22:56:52.043 INFO:teuthology.orchestra.run.smithi171.stdout:2 2024-09-17T22:56:52.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 28b3b7ca-d7d7-4ed9-8dfe-113d3579209c 3 2024-09-17T22:56:52.097 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:52.094+0000 7fb1be521040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T22:56:52.143 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:52.140+0000 7fb1be521040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T22:56:52.202 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.199+0000 7f27823d6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T22:56:52.238 INFO:teuthology.orchestra.run.smithi171.stdout:3 2024-09-17T22:56:52.249 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d379c252-e66a-4d0e-af0a-81f6bda48c08 4 2024-09-17T22:56:52.256 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.253+0000 7f27823d6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T22:56:52.293 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:52.290+0000 7fb1be521040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T22:56:52.433 INFO:teuthology.orchestra.run.smithi171.stdout:4 2024-09-17T22:56:52.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new e78a100f-1ff3-4af6-abd8-a069030ec951 5 2024-09-17T22:56:52.647 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.644+0000 7f27823d6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T22:56:52.668 INFO:teuthology.orchestra.run.smithi171.stdout:5 2024-09-17T22:56:52.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new fc1af848-dfaa-4e40-a0af-8982c70f7ea9 6 2024-09-17T22:56:52.737 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.733+0000 7f27823d6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T22:56:52.863 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.859+0000 7f27823d6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:56:52.878 INFO:teuthology.orchestra.run.smithi171.stdout:6 2024-09-17T22:56:52.889 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new ccd75a18-ded4-4f75-a09a-926da5362a7f 7 2024-09-17T22:56:52.915 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:52.911+0000 7f27823d6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T22:56:53.032 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.029+0000 7f27823d6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T22:56:53.071 INFO:teuthology.orchestra.run.smithi171.stdout:7 2024-09-17T22:56:53.082 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T22:56:53.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T22:56:53.084 INFO:tasks.ceph.osd.0:Started 2024-09-17T22:56:53.085 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T22:56:53.085 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T22:56:53.087 INFO:tasks.ceph.osd.1:Started 2024-09-17T22:56:53.087 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T22:56:53.087 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T22:56:53.088 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.085+0000 7f27823d6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T22:56:53.089 INFO:tasks.ceph.osd.2:Started 2024-09-17T22:56:53.089 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T22:56:53.089 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T22:56:53.092 INFO:tasks.ceph.osd.3:Started 2024-09-17T22:56:53.092 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T22:56:53.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T22:56:53.094 INFO:tasks.ceph.osd.4:Started 2024-09-17T22:56:53.094 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T22:56:53.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T22:56:53.096 INFO:tasks.ceph.osd.5:Started 2024-09-17T22:56:53.096 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T22:56:53.097 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T22:56:53.099 INFO:tasks.ceph.osd.6:Started 2024-09-17T22:56:53.099 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T22:56:53.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T22:56:53.101 INFO:tasks.ceph.osd.7:Started 2024-09-17T22:56:53.101 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:56:53.144 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.141+0000 7f27823d6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T22:56:53.271 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.268+0000 7f27823d6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T22:56:53.285 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:56:53.285 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":10,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:56:53.064036+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:56:53.290 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.287+0000 7fb1be521040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T22:56:53.296 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T22:56:53.296 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T22:56:53.320 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.317+0000 7f27823d6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T22:56:53.344 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.341+0000 7fb1be521040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T22:56:53.371 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.368+0000 7f27823d6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T22:56:53.438 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.435+0000 7f27823d6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T22:56:53.484 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.480+0000 7f27823d6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T22:56:53.529 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:53.526+0000 7f27823d6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T22:56:53.713 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.710+0000 7fb1be521040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T22:56:53.801 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.798+0000 7fb1be521040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T22:56:53.908 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.905+0000 7fb1be521040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:56:53.953 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:53.951+0000 7fb1be521040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T22:56:54.070 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.067+0000 7fb1be521040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T22:56:54.103 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:56:54.104 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:56:54.101+0000 7f57c242f640 -1 Falling back to public interface 2024-09-17T22:56:54.129 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.126+0000 7fb1be521040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T22:56:54.177 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.174+0000 7fb1be521040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T22:56:54.184 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:54.181+0000 7f27823d6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T22:56:54.266 INFO:teuthology.misc.health.smithi159.stdout: 2024-09-17T22:56:54.267 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":10,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:56:53.064036+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:56:54.276 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T22:56:54.299 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.297+0000 7fb1be521040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T22:56:54.317 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:56:54.314+0000 7fc088e14640 -1 Falling back to public interface 2024-09-17T22:56:54.351 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.349+0000 7fb1be521040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T22:56:54.404 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.400+0000 7fb1be521040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T22:56:54.471 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.468+0000 7fb1be521040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T22:56:54.514 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.512+0000 7fb1be521040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T22:56:54.557 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:56:54.553+0000 7fbccd20d640 -1 Falling back to public interface 2024-09-17T22:56:54.559 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:54.557+0000 7fb1be521040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T22:56:54.569 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:56:54.566+0000 7f4821f23640 -1 Falling back to public interface 2024-09-17T22:56:54.573 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:56:54.570+0000 7f673700b640 -1 Falling back to public interface 2024-09-17T22:56:54.790 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:56:54.787+0000 7f483de20640 -1 Falling back to public interface 2024-09-17T22:56:54.806 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:56:54.802+0000 7f2db06b6640 -1 Falling back to public interface 2024-09-17T22:56:55.040 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:56:55.036+0000 7f839ac13640 -1 Falling back to public interface 2024-09-17T22:56:55.196 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:55.193+0000 7f2722c9b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T22:56:55.196 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-17T22:56:55.193+0000 7f2722c9b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T22:56:55.228 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-09-17T22:56:55.225+0000 7fb1be521040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T22:56:57.858 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:56:57.855+0000 7f57c242f640 -1 osd.7 0 log_to_monitors true 2024-09-17T22:56:57.890 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:56:57.888+0000 7fc088e14640 -1 osd.1 0 log_to_monitors true 2024-09-17T22:56:58.097 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:56:58.095+0000 7f4821f23640 -1 osd.0 0 log_to_monitors true 2024-09-17T22:56:58.544 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:56:58.541+0000 7f483de20640 -1 osd.3 0 log_to_monitors true 2024-09-17T22:56:58.589 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:56:58.585+0000 7fbccd20d640 -1 osd.6 0 log_to_monitors true 2024-09-17T22:56:58.594 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:56:58.591+0000 7f673700b640 -1 osd.2 0 log_to_monitors true 2024-09-17T22:56:58.813 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:56:58.810+0000 7f2db06b6640 -1 osd.5 0 log_to_monitors true 2024-09-17T22:56:59.034 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:56:59.030+0000 7f839ac13640 -1 osd.4 0 log_to_monitors true 2024-09-17T22:57:00.129 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:00.126+0000 7f57be199640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T22:57:00.147 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:00.145+0000 7f57b57a5640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:00.734 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:00.731+0000 7f481deb2640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T22:57:00.788 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:00.785+0000 7fc0847e1640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T22:57:00.855 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:00.853+0000 7f4815cd2640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:00.881 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:00.879+0000 7fc07c601640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:01.082 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:57:01.455 INFO:teuthology.misc.health.smithi159.stdout: 2024-09-17T22:57:01.456 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":13,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:57:00.204245+0000","last_up_change":"2024-09-17T22:57:00.204245+0000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:57:01.469 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T22:57:01.756 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:01.753+0000 7f67329f9640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T22:57:01.766 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:01.764+0000 7f672a819640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:01.792 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:01.789+0000 7f483a020640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T22:57:01.803 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:01.801+0000 7f483162c640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:02.010 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:02.008+0000 7fbcc945f640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T22:57:02.022 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:02.020+0000 7f2dac645640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T22:57:02.026 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:02.024+0000 7fbcc0a6b640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:02.038 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:02.036+0000 7f2da4465640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:02.048 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:02.046+0000 7f8396506640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T22:57:02.065 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:02.063+0000 7f838e326640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:57:08.277 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:57:08.561 INFO:teuthology.misc.health.smithi159.stdout: 2024-09-17T22:57:08.561 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":17,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:57:04.856756+0000","last_up_change":"2024-09-17T22:57:02.843283+0000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6817","nonce":927278608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6819","nonce":927278608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6823","nonce":927278608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6821","nonce":927278608}]},"public_addr":"172.21.15.159:6817/927278608","cluster_addr":"172.21.15.159:6819/927278608","heartbeat_back_addr":"172.21.15.159:6823/927278608","heartbeat_front_addr":"172.21.15.159:6821/927278608","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6819","nonce":3064143709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6821","nonce":3064143709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6825","nonce":3064143709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6823","nonce":3064143709}]},"public_addr":"172.21.15.171:6819/3064143709","cluster_addr":"172.21.15.171:6821/3064143709","heartbeat_back_addr":"172.21.15.171:6825/3064143709","heartbeat_front_addr":"172.21.15.171:6823/3064143709","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:57:08.569 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T22:57:08.570 INFO:tasks.ceph:Creating RBD pool 2024-09-17T22:57:08.570 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T22:57:08.883 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'rbd' created 2024-09-17T22:57:08.896 DEBUG:teuthology.orchestra.run.smithi159:> rbd --cluster ceph pool init rbd 2024-09-17T22:57:11.909 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T22:57:11.910 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T22:57:11.910 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T22:57:11.910 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T22:57:11.910 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T22:57:12.247 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:12.255 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":6,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":3002909618},{"type":"v1","addr":"172.21.15.171:6809","nonce":3002909618}]},"active_addr":"172.21.15.171:6809/3002909618","active_change":"2024-09-17T22:56:54.184797+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":4049969496}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2236308530}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2094802130}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":3146562127}]}]} 2024-09-17T22:57:12.260 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T22:57:12.261 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T22:57:12.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:57:12.536 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:12.536 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:57:11.887022+0000","last_up_change":"2024-09-17T22:57:02.843283+0000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:57:08.859324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6817","nonce":927278608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6819","nonce":927278608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6823","nonce":927278608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6821","nonce":927278608}]},"public_addr":"172.21.15.159:6817/927278608","cluster_addr":"172.21.15.159:6819/927278608","heartbeat_back_addr":"172.21.15.159:6823/927278608","heartbeat_front_addr":"172.21.15.159:6821/927278608","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6819","nonce":3064143709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6821","nonce":3064143709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6825","nonce":3064143709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6823","nonce":3064143709}]},"public_addr":"172.21.15.171:6819/3064143709","cluster_addr":"172.21.15.171:6821/3064143709","heartbeat_back_addr":"172.21.15.171:6825/3064143709","heartbeat_front_addr":"172.21.15.171:6823/3064143709","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:57:12.548 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T22:57:12.549 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:57:12.824 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:12.824 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T22:57:11.887022+0000","last_up_change":"2024-09-17T22:57:02.843283+0000","last_in_change":"2024-09-17T22:56:53.064036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:57:08.859324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6817","nonce":927278608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6819","nonce":927278608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6823","nonce":927278608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":927278608},{"type":"v1","addr":"172.21.15.159:6821","nonce":927278608}]},"public_addr":"172.21.15.159:6817/927278608","cluster_addr":"172.21.15.159:6819/927278608","heartbeat_back_addr":"172.21.15.159:6823/927278608","heartbeat_front_addr":"172.21.15.159:6821/927278608","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6819","nonce":3064143709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6821","nonce":3064143709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6825","nonce":3064143709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":3064143709},{"type":"v1","addr":"172.21.15.171:6823","nonce":3064143709}]},"public_addr":"172.21.15.171:6819/3064143709","cluster_addr":"172.21.15.171:6821/3064143709","heartbeat_back_addr":"172.21.15.171:6825/3064143709","heartbeat_front_addr":"172.21.15.171:6823/3064143709","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:57:12.837 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T22:57:12.838 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T22:57:12.838 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T22:57:12.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T22:57:12.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T22:57:12.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T22:57:12.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T22:57:12.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T22:57:13.073 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-09-17T22:57:13.074 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T22:57:13.077 INFO:teuthology.orchestra.run.smithi159.stdout:64424509445 2024-09-17T22:57:13.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T22:57:13.081 INFO:teuthology.orchestra.run.smithi159.stdout:60129542150 2024-09-17T22:57:13.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T22:57:13.084 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-09-17T22:57:13.084 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T22:57:13.085 INFO:teuthology.orchestra.run.smithi159.stdout:55834574854 2024-09-17T22:57:13.085 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T22:57:13.087 INFO:teuthology.orchestra.run.smithi159.stdout:64424509445 2024-09-17T22:57:13.087 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T22:57:13.090 INFO:teuthology.orchestra.run.smithi159.stdout:60129542150 2024-09-17T22:57:13.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T22:57:13.097 INFO:teuthology.orchestra.run.smithi159.stdout:64424509445 2024-09-17T22:57:13.097 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T22:57:13.644 INFO:teuthology.orchestra.run.smithi159.stdout:60129542148 2024-09-17T22:57:13.659 INFO:teuthology.orchestra.run.smithi159.stdout:64424509444 2024-09-17T22:57:13.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542148 for osd.2 2024-09-17T22:57:13.672 INFO:teuthology.orchestra.run.smithi159.stdout:60129542148 2024-09-17T22:57:13.677 INFO:teuthology.orchestra.run.smithi159.stdout:60129542149 2024-09-17T22:57:13.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-17T22:57:13.688 INFO:teuthology.orchestra.run.smithi159.stdout:64424509444 2024-09-17T22:57:13.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.3 2024-09-17T22:57:13.689 INFO:teuthology.orchestra.run.smithi159.stdout:60129542148 2024-09-17T22:57:13.693 INFO:teuthology.orchestra.run.smithi159.stdout:55834574853 2024-09-17T22:57:13.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-09-17T22:57:13.700 INFO:teuthology.orchestra.run.smithi159.stdout:64424509444 2024-09-17T22:57:13.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-17T22:57:13.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.1 2024-09-17T22:57:13.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-09-17T22:57:13.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-09-17T22:57:14.667 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T22:57:14.681 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T22:57:14.690 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T22:57:14.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T22:57:14.708 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T22:57:14.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T22:57:14.714 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T22:57:14.716 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T22:57:15.241 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-09-17T22:57:15.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-09-17T22:57:15.261 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.283 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-09-17T22:57:15.292 INFO:teuthology.orchestra.run.smithi159.stdout:60129542150 2024-09-17T22:57:15.293 INFO:teuthology.orchestra.run.smithi159.stdout:60129542150 2024-09-17T22:57:15.296 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-09-17T22:57:15.298 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-09-17T22:57:15.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-17T22:57:15.299 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-17T22:57:15.317 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-09-17T22:57:15.318 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.6 2024-09-17T22:57:15.319 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.5 2024-09-17T22:57:15.320 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.326 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-09-17T22:57:15.326 INFO:teuthology.orchestra.run.smithi159.stdout:55834574854 2024-09-17T22:57:15.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-09-17T22:57:15.341 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.4 2024-09-17T22:57:15.341 DEBUG:teuthology.parallel:result is None 2024-09-17T22:57:15.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T22:57:15.341 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:15.656 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:15.657 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:15.669 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-17T22:57:14.193298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219048,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2854912,"data_stored":1745686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.315923"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.898422+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:11.898422+0000","last_peered":"2024-09-17T22:57:11.898422+0000","last_clean":"2024-09-17T22:57:11.898422+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:11.898422+0000","last_undegraded":"2024-09-17T22:57:11.898422+0000","last_fullsized":"2024-09-17T22:57:11.898422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901349+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:11.901349+0000","last_peered":"2024-09-17T22:57:11.901349+0000","last_clean":"2024-09-17T22:57:11.901349+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:11.901349+0000","last_undegraded":"2024-09-17T22:57:11.901349+0000","last_fullsized":"2024-09-17T22:57:11.901349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700962+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:12.700962+0000","last_peered":"2024-09-17T22:57:12.700962+0000","last_clean":"2024-09-17T22:57:12.700962+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:12.700962+0000","last_undegraded":"2024-09-17T22:57:12.700962+0000","last_fullsized":"2024-09-17T22:57:12.700962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901338+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:11.901338+0000","last_peered":"2024-09-17T22:57:11.901338+0000","last_clean":"2024-09-17T22:57:11.901338+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:11.901338+0000","last_undegraded":"2024-09-17T22:57:11.901338+0000","last_fullsized":"2024-09-17T22:57:11.901338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.902081+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:11.902081+0000","last_peered":"2024-09-17T22:57:11.902081+0000","last_clean":"2024-09-17T22:57:11.902081+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:11.902081+0000","last_undegraded":"2024-09-17T22:57:11.902081+0000","last_fullsized":"2024-09-17T22:57:11.902081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700316+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:12.700316+0000","last_peered":"2024-09-17T22:57:12.700316+0000","last_clean":"2024-09-17T22:57:12.700316+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:12.700316+0000","last_undegraded":"2024-09-17T22:57:12.700316+0000","last_fullsized":"2024-09-17T22:57:12.700316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700905+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:12.700905+0000","last_peered":"2024-09-17T22:57:12.700905+0000","last_clean":"2024-09-17T22:57:12.700905+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:12.700905+0000","last_undegraded":"2024-09-17T22:57:12.700905+0000","last_fullsized":"2024-09-17T22:57:12.700905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.899932+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:11.899932+0000","last_peered":"2024-09-17T22:57:11.899932+0000","last_clean":"2024-09-17T22:57:11.899932+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:11.899932+0000","last_undegraded":"2024-09-17T22:57:11.899932+0000","last_fullsized":"2024-09-17T22:57:11.899932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700547+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:12.700547+0000","last_peered":"2024-09-17T22:57:12.700547+0000","last_clean":"2024-09-17T22:57:12.700547+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:12.700547+0000","last_undegraded":"2024-09-17T22:57:12.700547+0000","last_fullsized":"2024-09-17T22:57:12.700547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":65511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:15.670 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:15.948 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:15.948 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:15.960 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-17T22:57:14.193298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219048,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2854912,"data_stored":1745686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.315923"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.898422+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:11.898422+0000","last_peered":"2024-09-17T22:57:11.898422+0000","last_clean":"2024-09-17T22:57:11.898422+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:11.898422+0000","last_undegraded":"2024-09-17T22:57:11.898422+0000","last_fullsized":"2024-09-17T22:57:11.898422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901349+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:11.901349+0000","last_peered":"2024-09-17T22:57:11.901349+0000","last_clean":"2024-09-17T22:57:11.901349+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:11.901349+0000","last_undegraded":"2024-09-17T22:57:11.901349+0000","last_fullsized":"2024-09-17T22:57:11.901349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700962+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:12.700962+0000","last_peered":"2024-09-17T22:57:12.700962+0000","last_clean":"2024-09-17T22:57:12.700962+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:12.700962+0000","last_undegraded":"2024-09-17T22:57:12.700962+0000","last_fullsized":"2024-09-17T22:57:12.700962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901338+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:11.901338+0000","last_peered":"2024-09-17T22:57:11.901338+0000","last_clean":"2024-09-17T22:57:11.901338+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:11.901338+0000","last_undegraded":"2024-09-17T22:57:11.901338+0000","last_fullsized":"2024-09-17T22:57:11.901338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.902081+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:11.902081+0000","last_peered":"2024-09-17T22:57:11.902081+0000","last_clean":"2024-09-17T22:57:11.902081+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:11.902081+0000","last_undegraded":"2024-09-17T22:57:11.902081+0000","last_fullsized":"2024-09-17T22:57:11.902081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700316+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:12.700316+0000","last_peered":"2024-09-17T22:57:12.700316+0000","last_clean":"2024-09-17T22:57:12.700316+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:12.700316+0000","last_undegraded":"2024-09-17T22:57:12.700316+0000","last_fullsized":"2024-09-17T22:57:12.700316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700905+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:12.700905+0000","last_peered":"2024-09-17T22:57:12.700905+0000","last_clean":"2024-09-17T22:57:12.700905+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:12.700905+0000","last_undegraded":"2024-09-17T22:57:12.700905+0000","last_fullsized":"2024-09-17T22:57:12.700905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.899932+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:11.899932+0000","last_peered":"2024-09-17T22:57:11.899932+0000","last_clean":"2024-09-17T22:57:11.899932+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:11.899932+0000","last_undegraded":"2024-09-17T22:57:11.899932+0000","last_fullsized":"2024-09-17T22:57:11.899932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700547+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:12.700547+0000","last_peered":"2024-09-17T22:57:12.700547+0000","last_clean":"2024-09-17T22:57:12.700547+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:12.700547+0000","last_undegraded":"2024-09-17T22:57:12.700547+0000","last_fullsized":"2024-09-17T22:57:12.700547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":65511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:15.961 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T22:57:15.961 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T22:57:15.961 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T22:57:15.961 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T22:57:16.271 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T22:57:16.283 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T22:57:16.284 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T22:57:16.295 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T22:57:16.295 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-17T22:57:16.295 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T22:57:16.295 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T22:57:16.591 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T22:57:16.591 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T22:57:16.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:16.863 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:16.864 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:16.876 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T22:57:16.193897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219048,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2854912,"data_stored":1745686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.314031"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.898422+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:11.898422+0000","last_peered":"2024-09-17T22:57:11.898422+0000","last_clean":"2024-09-17T22:57:11.898422+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:11.898422+0000","last_undegraded":"2024-09-17T22:57:11.898422+0000","last_fullsized":"2024-09-17T22:57:11.898422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901349+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:11.901349+0000","last_peered":"2024-09-17T22:57:11.901349+0000","last_clean":"2024-09-17T22:57:11.901349+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:11.901349+0000","last_undegraded":"2024-09-17T22:57:11.901349+0000","last_fullsized":"2024-09-17T22:57:11.901349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700962+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:12.700962+0000","last_peered":"2024-09-17T22:57:12.700962+0000","last_clean":"2024-09-17T22:57:12.700962+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:12.700962+0000","last_undegraded":"2024-09-17T22:57:12.700962+0000","last_fullsized":"2024-09-17T22:57:12.700962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901338+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:11.901338+0000","last_peered":"2024-09-17T22:57:11.901338+0000","last_clean":"2024-09-17T22:57:11.901338+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:11.901338+0000","last_undegraded":"2024-09-17T22:57:11.901338+0000","last_fullsized":"2024-09-17T22:57:11.901338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.902081+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:11.902081+0000","last_peered":"2024-09-17T22:57:11.902081+0000","last_clean":"2024-09-17T22:57:11.902081+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:11.902081+0000","last_undegraded":"2024-09-17T22:57:11.902081+0000","last_fullsized":"2024-09-17T22:57:11.902081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700316+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:12.700316+0000","last_peered":"2024-09-17T22:57:12.700316+0000","last_clean":"2024-09-17T22:57:12.700316+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:12.700316+0000","last_undegraded":"2024-09-17T22:57:12.700316+0000","last_fullsized":"2024-09-17T22:57:12.700316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700905+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:12.700905+0000","last_peered":"2024-09-17T22:57:12.700905+0000","last_clean":"2024-09-17T22:57:12.700905+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:12.700905+0000","last_undegraded":"2024-09-17T22:57:12.700905+0000","last_fullsized":"2024-09-17T22:57:12.700905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.899932+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:11.899932+0000","last_peered":"2024-09-17T22:57:11.899932+0000","last_clean":"2024-09-17T22:57:11.899932+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:11.899932+0000","last_undegraded":"2024-09-17T22:57:11.899932+0000","last_fullsized":"2024-09-17T22:57:11.899932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700547+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:12.700547+0000","last_peered":"2024-09-17T22:57:12.700547+0000","last_clean":"2024-09-17T22:57:12.700547+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:12.700547+0000","last_undegraded":"2024-09-17T22:57:12.700547+0000","last_fullsized":"2024-09-17T22:57:12.700547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":65511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:16.877 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:17.152 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:17.153 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:17.165 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T22:57:16.193897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219048,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2854912,"data_stored":1745686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.314031"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.898422+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:11.898422+0000","last_peered":"2024-09-17T22:57:11.898422+0000","last_clean":"2024-09-17T22:57:11.898422+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:11.898422+0000","last_undegraded":"2024-09-17T22:57:11.898422+0000","last_fullsized":"2024-09-17T22:57:11.898422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901349+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:11.901349+0000","last_peered":"2024-09-17T22:57:11.901349+0000","last_clean":"2024-09-17T22:57:11.901349+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:11.901349+0000","last_undegraded":"2024-09-17T22:57:11.901349+0000","last_fullsized":"2024-09-17T22:57:11.901349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700962+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:12.700962+0000","last_peered":"2024-09-17T22:57:12.700962+0000","last_clean":"2024-09-17T22:57:12.700962+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:12.700962+0000","last_undegraded":"2024-09-17T22:57:12.700962+0000","last_fullsized":"2024-09-17T22:57:12.700962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.901338+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:11.901338+0000","last_peered":"2024-09-17T22:57:11.901338+0000","last_clean":"2024-09-17T22:57:11.901338+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:11.901338+0000","last_undegraded":"2024-09-17T22:57:11.901338+0000","last_fullsized":"2024-09-17T22:57:11.901338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.902081+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:11.902081+0000","last_peered":"2024-09-17T22:57:11.902081+0000","last_clean":"2024-09-17T22:57:11.902081+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:11.902081+0000","last_undegraded":"2024-09-17T22:57:11.902081+0000","last_fullsized":"2024-09-17T22:57:11.902081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700316+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:12.700316+0000","last_peered":"2024-09-17T22:57:12.700316+0000","last_clean":"2024-09-17T22:57:12.700316+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:12.700316+0000","last_undegraded":"2024-09-17T22:57:12.700316+0000","last_fullsized":"2024-09-17T22:57:12.700316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700905+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:12.700905+0000","last_peered":"2024-09-17T22:57:12.700905+0000","last_clean":"2024-09-17T22:57:12.700905+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:12.700905+0000","last_undegraded":"2024-09-17T22:57:12.700905+0000","last_fullsized":"2024-09-17T22:57:12.700905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:11.899932+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:11.899932+0000","last_peered":"2024-09-17T22:57:11.899932+0000","last_clean":"2024-09-17T22:57:11.899932+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:11.899932+0000","last_undegraded":"2024-09-17T22:57:11.899932+0000","last_fullsized":"2024-09-17T22:57:11.899932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:57:12.700547+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:12.700547+0000","last_peered":"2024-09-17T22:57:12.700547+0000","last_clean":"2024-09-17T22:57:12.700547+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:12.700547+0000","last_undegraded":"2024-09-17T22:57:12.700547+0000","last_fullsized":"2024-09-17T22:57:12.700547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":65511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:17.166 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T22:57:17.166 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:epoch 21 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:fsid 33081d94-d15e-4bd4-ba4c-30022eb1be7a 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:created 2024-09-17T22:56:50.056303+0000 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:modified 2024-09-17T22:57:11.887022+0000 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:crush_version 5 2024-09-17T22:57:17.444 INFO:teuthology.orchestra.run.smithi159.stdout:full_ratio 0.95 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:backfillfull_ratio 0.9 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:nearfull_ratio 0.85 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:require_min_compat_client luminous 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:min_compat_client jewel 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:require_osd_release squid 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:stretch_mode_enabled false 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout: removed_snaps_queue [2~1] 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:max_osd 8 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/2087020665,v1:172.21.15.159:6809/2087020665] [v2:172.21.15.159:6810/2087020665,v1:172.21.15.159:6811/2087020665] exists,up 892321c3-409f-40e6-bfe1-c87f464d030f 2024-09-17T22:57:17.445 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/237398697,v1:172.21.15.159:6801/237398697] [v2:172.21.15.159:6802/237398697,v1:172.21.15.159:6803/237398697] exists,up cd777f15-a6ad-4f0c-820b-ef529dd9f008 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/927278608,v1:172.21.15.159:6817/927278608] [v2:172.21.15.159:6818/927278608,v1:172.21.15.159:6819/927278608] exists,up bd68e905-6e66-4735-9de8-66a99530420f 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/2987856795,v1:172.21.15.159:6825/2987856795] [v2:172.21.15.159:6826/2987856795,v1:172.21.15.159:6827/2987856795] exists,up 28b3b7ca-d7d7-4ed9-8dfe-113d3579209c 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6826/2123531187,v1:172.21.15.171:6827/2123531187] [v2:172.21.15.171:6828/2123531187,v1:172.21.15.171:6829/2123531187] exists,up d379c252-e66a-4d0e-af0a-81f6bda48c08 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6818/3064143709,v1:172.21.15.171:6819/3064143709] [v2:172.21.15.171:6820/3064143709,v1:172.21.15.171:6821/3064143709] exists,up e78a100f-1ff3-4af6-abd8-a069030ec951 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6810/1505991260,v1:172.21.15.171:6811/1505991260] [v2:172.21.15.171:6812/1505991260,v1:172.21.15.171:6813/1505991260] exists,up fc1af848-dfaa-4e40-a0af-8982c70f7ea9 2024-09-17T22:57:17.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/585405482,v1:172.21.15.171:6801/585405482] [v2:172.21.15.171:6802/585405482,v1:172.21.15.171:6803/585405482] exists,up ccd75a18-ded4-4f75-a09a-926da5362a7f 2024-09-17T22:57:17.457 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/2087020665,v1:172.21.15.159:6809/2087020665] [v2:172.21.15.159:6810/2087020665,v1:172.21.15.159:6811/2087020665] exists,up 892321c3-409f-40e6-bfe1-c87f464d030f', 'osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/237398697,v1:172.21.15.159:6801/237398697] [v2:172.21.15.159:6802/237398697,v1:172.21.15.159:6803/237398697] exists,up cd777f15-a6ad-4f0c-820b-ef529dd9f008', 'osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/927278608,v1:172.21.15.159:6817/927278608] [v2:172.21.15.159:6818/927278608,v1:172.21.15.159:6819/927278608] exists,up bd68e905-6e66-4735-9de8-66a99530420f', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/2987856795,v1:172.21.15.159:6825/2987856795] [v2:172.21.15.159:6826/2987856795,v1:172.21.15.159:6827/2987856795] exists,up 28b3b7ca-d7d7-4ed9-8dfe-113d3579209c', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6826/2123531187,v1:172.21.15.171:6827/2123531187] [v2:172.21.15.171:6828/2123531187,v1:172.21.15.171:6829/2123531187] exists,up d379c252-e66a-4d0e-af0a-81f6bda48c08', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6818/3064143709,v1:172.21.15.171:6819/3064143709] [v2:172.21.15.171:6820/3064143709,v1:172.21.15.171:6821/3064143709] exists,up e78a100f-1ff3-4af6-abd8-a069030ec951', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6810/1505991260,v1:172.21.15.171:6811/1505991260] [v2:172.21.15.171:6812/1505991260,v1:172.21.15.171:6813/1505991260] exists,up fc1af848-dfaa-4e40-a0af-8982c70f7ea9', 'osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/585405482,v1:172.21.15.171:6801/585405482] [v2:172.21.15.171:6802/585405482,v1:172.21.15.171:6803/585405482] exists,up ccd75a18-ded4-4f75-a09a-926da5362a7f'] 2024-09-17T22:57:17.457 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T22:57:17.554 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mon.a", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster": "ceph", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "admin_socket_mode": "", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_client_required": "cephx, none", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_cluster_required": "cephx", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_debug": "true", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_service_required": "cephx", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T22:57:17.555 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_supported": "cephx", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_aio": "true", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_aio_reap_max": "16", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_async_discard": "false", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_block_size": "4096", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_debug_aio": "false", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_enable_discard": "false", 2024-09-17T22:57:17.556 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_flock_retry": "3", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_inject_crash": "0", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_ioring": "false", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_ioring_hipri": "false", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T22:57:17.557 INFO:teuthology.orchestra.run.smithi159.stdout: "bdev_type": "", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_allocator": "hybrid", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_buffered_io": "true", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T22:57:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_replay_recovery": "false", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_sync_write": "false", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_allocator": "hybrid", 2024-09-17T22:57:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_create": "true", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_db_create": "false", 2024-09-17T22:57:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_db_path": "", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_db_size": "0", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_path": "", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_size": "107374182400", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_wal_create": "false", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_wal_path": "", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bluefs": "true", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T22:57:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_autotune": "true", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_size": "0", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T22:57:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_type": "2q", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_clone_cow": "true", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_mode": "none", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T22:57:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_freelist": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T22:57:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T22:57:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fail_eio": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T22:57:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_blob_size": "0", 2024-09-17T22:57:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T22:57:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_spdk_mem": "512", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T22:57:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_tracing": "false", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T22:57:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T22:57:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_cluster_require_version": "2", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_require_signatures": "false", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_require_version": "2", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_service_require_signatures": "false", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_service_require_version": "2", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephx_sign_messages": "true", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "chdir": "", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_acl_type": "", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_asio_thread_count": "2", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_cache_mid": "0.750000", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_cache_size": "16384", 2024-09-17T22:57:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_caps_release_delay": "5", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_check_pool_perm": "true", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_debug_force_sync_read": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_debug_getattr_caps": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_debug_inject_features": "", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_die_on_failed_remount": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_dirsize_rbytes": "true", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_force_lazyio": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_fs": "", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T22:57:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_inject_release_failure": "false", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_max_inline_size": "4096", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_mds_namespace": "", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_metadata": "", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_mount_gid": "-1", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_mount_timeout": "300", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_mount_uid": "-1", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_mountpoint": "/", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_notify_timeout": "10", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc": "true", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc_max_objects": "1000", 2024-09-17T22:57:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc_size": "209715200", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_permissions": "true", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_quota": "true", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_quota_df": "true", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_readahead_max_bytes": "0", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_readahead_max_periods": "4", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_readahead_min": "131072", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_reconnect_stale": "false", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_shutdown_timeout": "30", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_snapdir": ".snap", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_tick_interval": "1", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_trace": "", 2024-09-17T22:57:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_use_faked_inos": "false", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_use_random_mds": "false", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_graylog": "false", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_graylog_port": "12201", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_monitors": "default=true", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_syslog": "false", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "clog_to_syslog_level": "info", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "-", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_network": "", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_network_interface": "", 2024-09-17T22:57:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "compressor_zlib_isal": "false", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "compressor_zlib_level": "5", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "compressor_zstd_level": "1", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T22:57:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_location": "", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_location_hook": "", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_location_hook_timeout": "10", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "daemonize": "false", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "daos_pool": "tank", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_alienstore": "0/5", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_asok": "1/5", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_asok_assert_abort": "false", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T22:57:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_auth": "20/20", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_bdev": "1/3", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_bluefs": "1/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_bluestore": "1/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_buffer": "0/1", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_cephsqlite": "0/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_client": "0/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_compressor": "1/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_context": "0/1", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_crush": "1/1", 2024-09-17T22:57:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_crypto": "1/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_cyanstore": "0/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_dpdk": "1/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_eventtrace": "1/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_filer": "0/1", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_filestore": "1/3", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_finisher": "1/1", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_fuse": "1/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T22:57:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_javaclient": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_journal": "1/3", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_journaler": "0/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_kstore": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_lockdep": "0/1", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mclock": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_balancer": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_locker": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_log": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_migrator": "1/5", 2024-09-17T22:57:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_memstore": "1/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mgr": "2/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mgrc": "1/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_mon": "20/20", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_monc": "0/10", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_ms": "1/1", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_none": "0/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_objclass": "0/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_objectcacher": "0/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_objecter": "0/1", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_optracker": "0/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_osd": "1/5", 2024-09-17T22:57:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_paxos": "20/20", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_perfcounter": "1/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_prioritycache": "1/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rados": "0/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rbd": "0/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rbd_replay": "0/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_refs": "0/0", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_reserver": "1/1", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw": "1/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_access": "1/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T22:57:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_flight": "1/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rgw_sync": "1/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_rocksdb": "4/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_backref": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_cache": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_device": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_epm": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_journal": "0/5", 2024-09-17T22:57:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_lba": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_odata": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_omap": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_onode": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_t": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_seastore_tm": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_striper": "0/1", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_test": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_throttle": "1/1", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_timer": "0/1", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_tp": "0/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "debug_trace": "1/5", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T22:57:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "device_failure_prediction_mode": "none", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "err_to_graylog": "false", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "err_to_journald": "false", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "err_to_stderr": "true", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "err_to_syslog": "false", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event_tracing": "false", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_http_port": "9926", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_prio_limit": "5", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T22:57:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_sort_metrics": "true", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "exporter_stats_period": "5", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "fake_statfs_for_testing": "0", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "fatal_signal_handlers": "true", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filer_max_purge_ops": "10", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filer_max_truncate_ops": "128", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_blackhole": "false", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_btrfs_snap": "true", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_caller_concurrency": "10", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T22:57:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_debug_omap_check": "false", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_debug_verify_split": "false", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_dump_file": "", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fadvise": "true", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fail_eio": "true", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fd_cache_size": "128", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fiemap": "false", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T22:57:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_inject_stall": "0", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_journal_parallel": "false", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_journal_trailing": "false", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_journal_writeahead": "false", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_kill_at": "0", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T22:57:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_merge_threshold": "-10", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_odsync_write": "false", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_omap_backend_path": "", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T22:57:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_op_threads": "2", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_punch_hole": "false", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T22:57:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_queue_max_ops": "50", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_seek_data_hole": "false", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_sloppy_crc": "false", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_splice": "false", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_split_multiple": "2", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_split_rand_factor": "20", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_update_to": "1000", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T22:57:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_xfs_extsize": "false", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "filestore_zfs_snap": "false", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "fio_dir": "/tmp/fio", 2024-09-17T22:57:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "fsid": "33081d94-d15e-4bd4-ba4c-30022eb1be7a", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_allow_other": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_big_writes": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_debug": "false", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_default_permissions": "false", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_disable_pagecache": "false", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_max_write": "0", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_multithreaded": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_require_active_mds": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_set_user_groups": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_splice_move": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_splice_read": "true", 2024-09-17T22:57:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_splice_write": "true", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "gss_target_name": "ceph", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_file": "", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_inject_failure": "0", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_interval": "5", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "host": "smithi159", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T22:57:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "inject_early_sigterm": "false", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "jaeger_agent_port": "6799", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "jaeger_tracing_enable": "false", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_aio": "true", 2024-09-17T22:57:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_align_min_size": "65536", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_block_align": "true", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_block_size": "4096", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_dio": "true", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_discard": "false", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_force_aio": "false", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_ignore_corruption": "false", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_max_write_entries": "100", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_replay_from": "0", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T22:57:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_write_header_frequency": "0", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "journal_zero_on_create": "false", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "journaler_prefetch_periods": "10", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "journaler_prezero_periods": "5", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "journaler_write_head_interval": "15", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "keyfile": "", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_backend": "rocksdb", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T22:57:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_max_bytes": "67108864", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_max_ops": "512", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_onode_map_size": "1024", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "kstore_sync_transaction": "false", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "librados_thread_count": "2", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "lockdep": "false", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "lockdep_force_backtrace": "false", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "log_coarse_timestamps": "true", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "log_flush_on_exit": "false", 2024-09-17T22:57:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_graylog_port": "12201", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_max_new": "1000", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_max_recent": "10000", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_stderr_prefix": "", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": "true", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_graylog": "false", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_journald": "false", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_stderr": "false", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_syslog": "false", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T22:57:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_action_on_write_error": "1", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_alternate_name_max": "8192", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_asio_thread_count": "2", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_export_pin": "true", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_interval": "10", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_max": "-1", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_max_until": "-1", 2024-09-17T22:57:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_merge_size": "50", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_mode": "0", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_split_bits": "3", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_split_size": "10000", 2024-09-17T22:57:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_mid": "0.700000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T22:57:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_trim_interval": "1", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T22:57:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_debug_auth_pins": "false", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_debug_frag": "false", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_debug_scatterstat": "false", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_debug_subtrees": "false", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_default_dir_hash": "2", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_defer_session_stale": "true", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dir_max_entries": "0", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dir_prefetch": "true", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T22:57:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_early_reply": "true", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_enable_op_tracker": "true", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_enforce_unique_name": "true", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T22:57:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_health_dummy": "false", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T22:57:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_join_fs": "", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_journal_format": "1", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_create_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_export_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_import_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_journal_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_link_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_openc_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_rename_at": "0", 2024-09-17T22:57:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_max_events": "-1", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_max_segments": "128", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_pause": "false", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_segment_size": "0", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_trim_threshold": "128", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T22:57:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_completed_requests": "100000", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_export_size": "20971520", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_file_recover": "32", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_purge_files": "64", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_purge_ops": "8192", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_metrics_update_interval": "2", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_min_caps_per_client": "100", 2024-09-17T22:57:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_numa_node": "-1", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_history_duration": "600", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_history_size": "20", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_op_log_threshold": "5", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_ping_grace": "15", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_ping_interval": "5", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T22:57:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_max_caps": "30000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_replay_interval": "1.000000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T22:57:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_root_ino_gid": "0", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_root_ino_uid": "0", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_shutdown_check": "0", 2024-09-17T22:57:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_skip_ino": "0", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_snap_min_uid": "0", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_snap_rstat": "false", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_symlink_recovery": "true", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_thrash_exports": "0", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_thrash_fragments": "0", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_tick_interval": "5.000000", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_valgrind_exit": "false", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_verify_backtrace": "1", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_verify_scatter": "false", 2024-09-17T22:57:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_wipe_sessions": "false", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool_debug": "false", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "memstore_page_set": "false", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "memstore_page_size": "65536", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_client_bytes": "134217728", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_client_messages": "512", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T22:57:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_disabled_modules": "", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_mds_messages": "128", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_mon_messages": "128", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_history_duration": "600", 2024-09-17T22:57:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_history_size": "20", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_op_log_threshold": "5", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_osd_messages": "8192", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_pool": "true", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_standby_modules": "true", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_stats_period": "5", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_stats_threshold": "5", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_tick_period": "2", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T22:57:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_allow_pool_delete": "true", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_bytes": "104857600", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T22:57:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_client_target_rank": "-1", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_file_level": "debug", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T22:57:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_compact_on_start": "false", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_compact_on_trim": "true", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_cpu_threads": "4", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T22:57:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_data_avail_crit": "5", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_data_avail_warn": "5", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_dump_json": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_extra_checks": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T22:57:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_down_uptime_grace": "60", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_election_default_strategy": "1", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_election_timeout": "5.000000", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_enable_op_tracker": "true", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_fake_pool_delete": "false", 2024-09-17T22:57:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_force_quorum_join": "false", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_log_update_period": "5", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_max_detail": "50", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_to_clog": "true", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_host": "172.21.15.159,[v2:172.21.15.159:3301,v1:172.21.15.159:6790],172.21.15.171", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_host_override": "", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_initial_members": "", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T22:57:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_lease": "5.000000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_log_full_interval": "50", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_log_max": "10000", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_log_max_summary": "50", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_log_epochs": "500", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T22:57:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_osd": "10000", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_memory_autotune": "true", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_memory_target": "2147483648", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_digest_period": "5", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T22:57:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_complaint_time": "30", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_history_duration": "600", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_history_size": "20", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_op_log_threshold": "5", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T22:57:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_cache_size": "500", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T22:57:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_report_timeout": "900", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T22:57:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T22:57:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_reweight_max_osds": "4", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_scrub_interval": "86400", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_scrub_max_keys": "100", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_scrub_timeout": "300", 2024-09-17T22:57:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_session_timeout": "300", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_smart_report_timeout": "5", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_stretch_pool_size": "4", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_debug": "false", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T22:57:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_tick_interval": "5", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T22:57:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_older_version": "true", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T22:57:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "monmap": "", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "motr_tracing_enabled": "false", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_op_threads": "3", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_cm": "false", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_device_name": "", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T22:57:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_sl": "3", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_rdma_type": "ib", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_async_reap_threshold": "5", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_before_connect": "false", 2024-09-17T22:57:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_ipv4": "true", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_ipv6": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_msgr1": "true", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_msgr2": "true", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_port_max": "7568", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_port_min": "6800", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_retry_count": "3", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_bind_retry_delay": "5", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_blackhole_client": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_blackhole_mds": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_blackhole_mgr": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_blackhole_mon": "false", 2024-09-17T22:57:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_blackhole_osd": "false", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_client_mode": "crc secure", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_cluster_type": "", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_compress_secure": "false", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_crc_data": "true", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_crc_header": "true", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_die_on_bug": "true", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_die_on_old_message": "true", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T22:57:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_hugepages": "", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_lro": "true", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T22:57:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_pmd": "", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_port_id": "0", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_dump_on_send": "false", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_delay_type": "", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_network_congestion": "0", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T22:57:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_max_accept_failures": "4", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_max_backoff": "15.000000", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_osd_compress_mode": "none", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_pq_min_cost": "65536", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_public_type": "", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_service_mode": "crc secure", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T22:57:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_tcp_nodelay": "true", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "ms_type": "async+posix", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "no_config_file": "false", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_inflight_ops": "1024", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter_timeout": "10.000000", 2024-09-17T22:57:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "objectstore_blackhole": "false", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "openssl_engine_opts": "", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_max_ops": "4", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_agent_slop": "0.020000", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_asio_thread_count": "2", 2024-09-17T22:57:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backfill_scan_max": "512", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backfill_scan_min": "64", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backoff_on_peering": "false", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_beacon_report_interval": "300", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bench_duration": "30", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T22:57:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_blkin_trace_all": "false", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_class_update_on_start": "true", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_client_message_cap": "256", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T22:57:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_client_op_priority": "63", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_client_watch_timeout": "30", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_command_max_records": "256", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_compact_on_start": "false", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_crush_update_on_start": "true", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T22:57:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_op_order": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T22:57:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_shutdown": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_trim_objects": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T22:57:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_default_notify_timeout": "30", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T22:57:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_enable_op_tracker": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_fast_info": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_fast_shutdown": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T22:57:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_function_tracing": "false", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_grace": "20", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_interval": "6", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_stale": "600", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T22:57:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_journal_size": "5120", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_kill_backfill_at": "0", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_map_cache_size": "50", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_map_dedup": "true", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_map_message_max": "40", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_attr_name_len": "100", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_attr_size": "0", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_backfills": "1", 2024-09-17T22:57:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_markdown_count": "5", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_markdown_period": "600", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_object_name_len": "2048", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_object_size": "134217728", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_pgls": "1024", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_push_cost": "8388608", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_push_objects": "10", 2024-09-17T22:57:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_scrubs": "3", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_max_write_size": "90", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T22:57:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_profile": "balanced", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_base": "805306368", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T22:57:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_target": "4294967296", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_target_autotune": "false", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_min_recovery_priority": "0", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mon_report_interval": "5", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_num_cache_shards": "32", 2024-09-17T22:57:17.655 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_numa_node": "-1", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_objecter_finishers": "1", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_objectstore": "bluestore", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_objectstore_fuse": "false", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_objectstore_tracing": "false", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_history_duration": "600", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_history_size": "20", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T22:57:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_log_threshold": "5", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_shards": "0", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T22:57:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_op_thread_timeout": "15", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_open_classes_on_start": "true", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_os_flags": "0", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_peering_op_priority": "255", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_delete_priority": "5", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T22:57:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_crimson": "false", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T22:57:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_flags": "0", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_min_size": "0", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_size": "2", 2024-09-17T22:57:17.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_default_type": "replicated", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_cost": "20971520", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_active": "0", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T22:57:17.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_op_priority": "3", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_priority": "5", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_repair_during_recovery": "false", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T22:57:17.662 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_cost": "52428800", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_end_hour": "0", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T22:57:17.663 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_priority": "5", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_skip_data_digest": "false", 2024-09-17T22:57:17.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_smart_report_timeout": "5", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_priority": "5", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_target_transaction_size": "30", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T22:57:17.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_tracing": "false", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_use_stale_snap": "false", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_kill_at": "0", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_max_join_drift": "10", 2024-09-17T22:57:17.666 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_min": "500", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_min_wait": "0.050000", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_service_trim_max": "500", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_service_trim_min": "250", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_stash_full_interval": "25", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_trim_max": "500", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "paxos_trim_min": "250", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "perf": "true", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T22:57:17.667 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "-", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrv": "", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "public_bind_addr": "-", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "public_network": "", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "public_network_interface": "", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "qat_compressor_enabled": "false", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rados_mon_op_timeout": "0", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rados_osd_op_timeout": "0", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rados_tracing": "false", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_atime_update_interval": "60", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T22:57:17.668 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache": "true", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_policy": "writearound", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_size": "33554432", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T22:57:17.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_compression_hint": "none", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_clone_format": "auto", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_data_pool": "", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_features": "61", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_format": "2", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_map_options": "", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_order": "22", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_pool": "rbd", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_stripe_count": "0", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T22:57:17.670 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_io_scheduler": "simple", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T22:57:17.671 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_order": "24", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_pool": "", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_journal_splay_width": "4", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T22:57:17.672 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T22:57:17.673 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T22:57:17.674 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_op_threads": "1", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_plugins": "", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T22:57:17.675 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T22:57:17.676 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_tracing": "false", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_validate_names": "true", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_validate_pool": "true", 2024-09-17T22:57:17.677 INFO:teuthology.orchestra.run.smithi159.stdout: "restapi_base_url": "", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "restapi_log_level": "", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_admin_entry": "admin", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_backend_store": "rados", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_barbican_url": "", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_beast_enable_async": "true", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T22:57:17.678 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_cache_enabled": "true", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_config_store": "rados", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_content_length_compat": "false", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T22:57:17.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_cross_domain_policy": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T22:57:17.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T22:57:17.681 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T22:57:17.682 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T22:57:17.683 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_d4n_port": "6379", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_log_window": "30", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T22:57:17.684 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T22:57:17.685 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dns_name": "", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dns_s3website_name": "", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_ops_log": "false", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_static_website": "false", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enable_usage_log": "false", 2024-09-17T22:57:17.686 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_expose_bucket": "false", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_extended_http_attrs": "", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_filter": "none", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_objs": "32", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T22:57:17.687 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_init_timeout": "300", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T22:57:17.688 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_password": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_project": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_token": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_admin_user": "", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_api_version": "2", 2024-09-17T22:57:17.689 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_url": "", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T22:57:17.690 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_max_objs": "32", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_max_worker": "3", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ldap_uri": "", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T22:57:17.691 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_log_http_headers": "", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_attr_size": "0", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T22:57:17.692 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_listing_results": "1000", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_notify_retries": "10", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T22:57:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T22:57:17.694 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_num_control_oids": "8", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_numa_node": "-1", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T22:57:17.695 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_op_tracing": "false", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_opa_token": "", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_opa_url": "", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ops_log_rados": "false", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T22:57:17.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_print_continue": "true", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T22:57:17.697 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_rados_tracing": "false", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_realm": "", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_realm_id": "", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_region": "", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_request_uri": "", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T22:57:17.698 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_resolve_cname": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_run_sync_thread": "true", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T22:57:17.699 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_script_uri": "", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_service_provider_name": "", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_client_id": "", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_client_secret": "", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_entry": "sts", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_key": "sts", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T22:57:17.700 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_auth_url": "", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_custom_header": "", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_need_stats": "true", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_tenant_name": "", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_url": "", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T22:57:17.701 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_lease_period": "120", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_thread_pool_size": "512", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T22:57:17.702 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_comment": "", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_createby": "", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_encoding": "", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_flag": "false", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_origin": "", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_torrent_tracker": "", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T22:57:17.703 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_usage_max_shards": "32", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_use_opa_authz": "false", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_counters_cache": "false", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T22:57:17.704 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_user_unique_email": "true", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_verify_ssl": "true", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zone": "", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zone_id": "", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zonegroup": "", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zonegroup_id": "", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_block_size": "4096", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T22:57:17.705 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T22:57:17.706 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_partition_filters": "false", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_perf": "false", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "run_dir": "/var/run/ceph", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_block_create": "true", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T22:57:17.707 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_device_size": "53687091200", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_full_integrity_check": "false", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_main_device_type": "SSD", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T22:57:17.708 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "seastore_segment_size": "67108864", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "set_keepcaps": "false", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "setgroup": "", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "setuser": "", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "setuser_match_path": "", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "thp": "false", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "threadpool_default_timeout": "60", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout: "throttler_perf_counter": "true" 2024-09-17T22:57:17.709 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:17.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T22:57:17.824 INFO:teuthology.orchestra.run.smithi159.stdout:mon.a: {} 2024-09-17T22:57:17.824 INFO:teuthology.orchestra.run.smithi159.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:57:17.834 INFO:teuthology.orchestra.run.smithi159.stdout:mon.b: {} 2024-09-17T22:57:17.834 INFO:teuthology.orchestra.run.smithi159.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:57:17.842 INFO:teuthology.orchestra.run.smithi159.stdout:mon.c: {} 2024-09-17T22:57:17.842 INFO:teuthology.orchestra.run.smithi159.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:57:17.853 DEBUG:teuthology.orchestra.run.smithi159:> type ceph-objectstore-tool 2024-09-17T22:57:17.868 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:57:17.868 DEBUG:teuthology.orchestra.run.smithi171:> type ceph-objectstore-tool 2024-09-17T22:57:17.887 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:57:17.887 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T22:57:17.899 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T22:57:17.900 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T22:57:17.900 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:17.900 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:17.900 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T22:57:17.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T22:57:17.901 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T22:57:17.901 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T22:57:17.901 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:17.901 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T22:57:17.902 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:17.902 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T22:57:17.902 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:57:17.902 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-17T22:57:17.910 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:17.907+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:17.949 INFO:teuthology.orchestra.run.smithi159.stdout:mapped writes DISABLED 2024-09-17T22:57:17.949 INFO:teuthology.orchestra.run.smithi159.stdout:mapped reads DISABLED 2024-09-17T22:57:17.949 INFO:teuthology.orchestra.run.smithi159.stdout:Seed set to 461 2024-09-17T22:57:18.003 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:18.001+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:18.027 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:18.028 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:18.042 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:18.050 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:18.051 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:18.068 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:18.069 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:18.079 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T22:57:18.102 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:18.103 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:18.104 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:18.102+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:18.120 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:18.120 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:18.160 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.160 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.169 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:18.210 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:18.208+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:18.244 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-17T22:57:18.314 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.314 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.314 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:18.312+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:18.323 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:18.339 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:18.339 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:18.356 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:18.356 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:18.374 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:18.374 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:18.391 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:18.407 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:18.405+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:18.426 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T22:57:18.451 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.451 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.460 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:18.545 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:18.543+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:18.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T22:57:18.610 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.610 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.610 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:18.607+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:18.619 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:18.709 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:18.707+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:18.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T22:57:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.777 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:18.811 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:18.808+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:18.884 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-17T22:57:18.911 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:18.909+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:18.925 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:18.925 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:18.935 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:18.940 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:57:19.012 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:19.010+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:19.028 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-17T22:57:19.049 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.049 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.058 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:19.113 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:19.111+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:19.205 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.205 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.214 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:19.214 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:19.211+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:19.315 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:19.312+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:19.359 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.359 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.368 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:19.415 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:19.413+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:19.511 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.512 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.516 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:19.514+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:19.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:19.617 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:19.615+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:19.625 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.625 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:19.718 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:19.716+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:19.751 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.751 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.760 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:19.819 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:19.816+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:19.913 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:19.913 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:19.919 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:19.917+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:19.921 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:20.020 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:20.018+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:20.025 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:20.026 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:20.035 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:20.122 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:20.119+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:20.138 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:20.139 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:20.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:20.232 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:20.230+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:20.290 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:20.291 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:20.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:20.340 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:20.338+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:20.405 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:20.443 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:20.443 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:20.449 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:20.447+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:20.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:20.524 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:20.521+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:20.526 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:20.526 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:20.542 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:20.542 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:20.555 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:20.556 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:20.558 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:20.558 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:20.564 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:20.573 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:20.573 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:20.589 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:20.606 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:20.606 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:20.621 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:20.621 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:20.624 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:20.622+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:20.638 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:20.638 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:20.670 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:20.670 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:20.670 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:20.670 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:20.670 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:20.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:20.761 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:20.759+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:20.824 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:20.824 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:20.825 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:57:20.825 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:20.827 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:20.824+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:20.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:20.928 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:20.926+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:20.956 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:57:20.974 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:20.975 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:20.975 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:20.983 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:21.030 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:21.027+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:21.127 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:21.128 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:21.131 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:21.128+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:21.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:21.231 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:21.229+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:21.241 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:21.241 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.377037137999999, 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:57:21.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:57:21.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.227446+0000", 2024-09-17T22:57:21.249 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.010192224000001, 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070451649999999999, 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 68 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227446+0000", 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227446+0000", 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227447+0000", 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227450+0000", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.478e-06 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227459+0000", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0229999999999998e-06 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227463+0000", 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0570000000000003e-06 2024-09-17T22:57:21.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227935+0000", 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047119499999999999 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.228016+0000", 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0951000000000005e-05 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.228132+0000", 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011661400000000001 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.231934+0000", 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038019199999999999 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234327+0000", 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002393217 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234415+0000", 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8075e-05 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234492+0000", 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6157999999999998e-05 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.255 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.227474+0000", 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.010164197000002, 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073300930000000002, 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 69 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227474+0000", 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227474+0000", 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227475+0000", 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3799999999999999e-07 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227508+0000", 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3439999999999998e-05 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227516+0000", 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8390000000000007e-06 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227520+0000", 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000004e-06 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.228459+0000", 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093929300000000001 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.228507+0000", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7419000000000001e-05 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.228582+0000", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5455000000000002e-05 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.231969+0000", 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0033873309999999999 2024-09-17T22:57:21.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234749+0000", 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027792120000000001 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234779+0000", 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0306000000000001e-05 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234805+0000", 2024-09-17T22:57:21.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5619999999999999e-05 2024-09-17T22:57:21.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.227528+0000", 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.010110499, 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007352333, 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 70 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227528+0000", 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227528+0000", 2024-09-17T22:57:21.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227529+0000", 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227532+0000", 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.151e-06 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227537+0000", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8919999999999999e-06 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.227540+0000", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.999e-06 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.231396+0000", 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038565259999999999 2024-09-17T22:57:21.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.231475+0000", 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9098999999999996e-05 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.231553+0000", 2024-09-17T22:57:21.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8127000000000006e-05 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.232562+0000", 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001008463 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234813+0000", 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022513089999999999 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234853+0000", 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9678000000000002e-05 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.234880+0000", 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7679e-05 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.249673+0000", 2024-09-17T22:57:21.269 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.987965336999999, 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077613420000000001, 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 86 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249673+0000", 2024-09-17T22:57:21.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249673+0000", 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249674+0000", 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:57:21.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249676+0000", 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1270000000000001e-06 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249680+0000", 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7809999999999996e-06 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249683+0000", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.853e-06 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.251560+0000", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001876754 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.251606+0000", 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5917000000000001e-05 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.251679+0000", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3362999999999995e-05 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.253240+0000", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001561142 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257370+0000", 2024-09-17T22:57:21.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041300379999999999 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257408+0000", 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7234999999999998e-05 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257435+0000", 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6889e-05 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.249691+0000", 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.987947863999999, 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078427440000000005, 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 87 2024-09-17T22:57:21.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249691+0000", 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249691+0000", 2024-09-17T22:57:21.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249691+0000", 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249754+0000", 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3440000000000002e-05 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249761+0000", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249765+0000", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6490000000000001e-06 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.252029+0000", 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002264 2024-09-17T22:57:21.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.252077+0000", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7933000000000003e-05 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.252223+0000", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014619700000000001 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.253866+0000", 2024-09-17T22:57:21.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001642608 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257478+0000", 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036127120000000001 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257510+0000", 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1597000000000003e-05 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257533+0000", 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3422e-05 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.249773+0000", 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.987865248999999, 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078357870000000003, 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 88 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249773+0000", 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249773+0000", 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249774+0000", 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249776+0000", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6199999999999999e-06 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249780+0000", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7400000000000002e-06 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.249784+0000", 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7230000000000002e-06 2024-09-17T22:57:21.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.253262+0000", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034782580000000001 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.253315+0000", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3177000000000003e-05 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.253389+0000", 2024-09-17T22:57:21.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.381e-05 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.254160+0000", 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077072999999999996 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257563+0000", 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003403381 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257589+0000", 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6092e-05 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.257609+0000", 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9796000000000002e-05 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371245+0000", 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866393633000001, 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067099839999999996, 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 110 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371245+0000", 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371245+0000", 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371246+0000", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371250+0000", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2969999999999997e-06 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371259+0000", 2024-09-17T22:57:21.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3000000000000007e-06 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371265+0000", 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6280000000000002e-06 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374224+0000", 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029591800000000001 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374301+0000", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6674999999999993e-05 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374428+0000", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012682599999999999 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.375718+0000", 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012905379999999999 2024-09-17T22:57:21.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377891+0000", 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021732969999999998 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377917+0000", 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6032000000000001e-05 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377955+0000", 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7579000000000001e-05 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371286+0000", 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866352194000001, 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067322179999999999, 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.295 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 111 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371286+0000", 2024-09-17T22:57:21.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371286+0000", 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371287+0000", 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371291+0000", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3959999999999999e-06 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371300+0000", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9150000000000002e-06 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371306+0000", 2024-09-17T22:57:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6049999999999997e-06 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374783+0000", 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034770299999999999 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374830+0000", 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6949000000000001e-05 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.374904+0000", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4078999999999993e-05 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376447+0000", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00154313 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377961+0000", 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001513676 2024-09-17T22:57:21.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377984+0000", 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3453999999999999e-05 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378019+0000", 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4413999999999999e-05 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371324+0000", 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866314333999998, 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067552369999999999, 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 112 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371324+0000", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371324+0000", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371325+0000", 2024-09-17T22:57:21.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2900000000000003e-07 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371409+0000", 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3508000000000001e-05 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371420+0000", 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1219999999999999e-05 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371426+0000", 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2689999999999996e-06 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.375132+0000", 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037058450000000001 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.375179+0000", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7043999999999999e-05 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.375251+0000", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2410000000000006e-05 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376486+0000", 2024-09-17T22:57:21.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012350289999999999 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378024+0000", 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015381710000000001 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378048+0000", 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3472000000000002e-05 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378080+0000", 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.154e-05 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371455+0000", 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866184060999998, 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067307479999999999, 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.308 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 114 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371455+0000", 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371455+0000", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371455+0000", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371463+0000", 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1440000000000003e-06 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371476+0000", 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3266e-05 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371480+0000", 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.873e-06 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376110+0000", 2024-09-17T22:57:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0046303680000000002 2024-09-17T22:57:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376155+0000", 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5065000000000003e-05 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376227+0000", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1618000000000004e-05 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377842+0000", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016145339999999999 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378146+0000", 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000304512 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378168+0000", 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2123000000000001e-05 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378185+0000", 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7025e-05 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371490+0000", 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866148739, 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067400680000000001, 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 115 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371490+0000", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371490+0000", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371490+0000", 2024-09-17T22:57:21.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371537+0000", 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6329999999999999e-05 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371543+0000", 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2040000000000002e-06 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371547+0000", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2150000000000003e-06 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376515+0000", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049677530000000001 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376569+0000", 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4156000000000001e-05 2024-09-17T22:57:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.376643+0000", 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4451999999999998e-05 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377884+0000", 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012405719999999999 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378193+0000", 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030939599999999998 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378214+0000", 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0219000000000001e-05 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378230+0000", 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6371999999999998e-05 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371553+0000", 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866085291000001, 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0088376449999999999, 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 116 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371553+0000", 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371553+0000", 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371554+0000", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9999999999999999e-07 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371556+0000", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6029999999999999e-06 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371561+0000", 2024-09-17T22:57:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9400000000000001e-06 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371565+0000", 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5659999999999998e-06 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377049+0000", 2024-09-17T22:57:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005483908 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377135+0000", 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6110000000000001e-05 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.377280+0000", 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000145255 2024-09-17T22:57:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378234+0000", 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00095376399999999996 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380319+0000", 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020851509999999999 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380351+0000", 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.163e-05 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380391+0000", 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0417999999999998e-05 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.371569+0000", 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.866069722999999, 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089181859999999998, 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 117 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371569+0000", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371569+0000", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371569+0000", 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.29e-07 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371571+0000", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3269999999999999e-06 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371575+0000", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4759999999999998e-06 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.371582+0000", 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8770000000000004e-06 2024-09-17T22:57:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378257+0000", 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0066755579999999998 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378308+0000", 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0543e-05 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378385+0000", 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7373999999999995e-05 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.378801+0000", 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000416174 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380424+0000", 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016225949999999999 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380452+0000", 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8277999999999999e-05 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.380487+0000", 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4755000000000002e-05 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.334 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778193+0000", 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.459446065000002, 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074241469999999999, 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 136 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778193+0000", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778193+0000", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778193+0000", 2024-09-17T22:57:21.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778195+0000", 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1220000000000002e-06 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778202+0000", 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3869999999999997e-06 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778205+0000", 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.781489+0000", 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003283841 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.781530+0000", 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0852999999999997e-05 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.781598+0000", 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7797000000000002e-05 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784573+0000", 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029757780000000001 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785547+0000", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097387099999999996 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785577+0000", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9779000000000001e-05 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785617+0000", 2024-09-17T22:57:21.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9625000000000001e-05 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778209+0000", 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.459429204999999, 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075021970000000004, 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 137 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778209+0000", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778209+0000", 2024-09-17T22:57:21.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778210+0000", 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778260+0000", 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0866999999999999e-05 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778269+0000", 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6100000000000006e-06 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778273+0000", 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4929999999999998e-06 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.781923+0000", 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036502800000000001 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.781972+0000", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8678999999999999e-05 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.782038+0000", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6398999999999999e-05 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785510+0000", 2024-09-17T22:57:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0034720290000000002 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785654+0000", 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014377899999999999 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785683+0000", 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9694000000000001e-05 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785712+0000", 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000002e-05 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778279+0000", 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.459359443, 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087027920000000009, 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 138 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778279+0000", 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778279+0000", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778280+0000", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778282+0000", 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:57:21.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778287+0000", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7570000000000004e-06 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778290+0000", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6850000000000001e-06 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784208+0000", 2024-09-17T22:57:21.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059185219999999998 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784251+0000", 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2892999999999997e-05 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784318+0000", 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7142000000000002e-05 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785751+0000", 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001432794 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.786929+0000", 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001178194 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.786957+0000", 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7990000000000001e-05 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.786982+0000", 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4820999999999998e-05 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778294+0000", 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.459344957999999, 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011672801, 2024-09-17T22:57:21.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 139 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778294+0000", 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778294+0000", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778294+0000", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778297+0000", 2024-09-17T22:57:21.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6460000000000002e-06 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778301+0000", 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6759999999999998e-06 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778304+0000", 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6029999999999999e-06 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784599+0000", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062953530000000001 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784644+0000", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4404999999999999e-05 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784719+0000", 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5383000000000007e-05 2024-09-17T22:57:21.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.786164+0000", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001444945 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789869+0000", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037050149999999999 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789942+0000", 2024-09-17T22:57:21.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2621999999999997e-05 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789966+0000", 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4947e-05 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778310+0000", 2024-09-17T22:57:21.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.459328568, 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011704473999999999, 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 140 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778310+0000", 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778310+0000", 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778310+0000", 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778359+0000", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8934000000000003e-05 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778368+0000", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.585e-06 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778373+0000", 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:57:21.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784946+0000", 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0065729960000000002 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.784992+0000", 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6471999999999998e-05 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785062+0000", 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9538999999999999e-05 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.786190+0000", 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011284209999999999 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789973+0000", 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037824960000000002 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789997+0000", 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4263000000000001e-05 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.790015+0000", 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7391e-05 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:04.778384+0000", 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 16.4592548, 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011673742000000001, 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 141 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778384+0000", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778384+0000", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778384+0000", 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778390+0000", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7679999999999997e-06 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778398+0000", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1149999999999994e-06 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.778409+0000", 2024-09-17T22:57:21.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1113e-05 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785781+0000", 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0073716449999999996 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785834+0000", 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3162999999999999e-05 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:21.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.785906+0000", 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1884000000000005e-05 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.789113+0000", 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032070639999999999 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.790018+0000", 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090538999999999995 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.790040+0000", 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1495e-05 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.790058+0000", 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7615999999999999e-05 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:21.371 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:21.373 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:21.371+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:21.373 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:21.474 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:21.472+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:21.476 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:21.477 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:21.485 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:21.574 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:21.572+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:21.588 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:21.589 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:21.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:21.676 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:21.673+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:21.702 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:21.702 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:21.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:21.702 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:21.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.199945+0000", 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.499121802000001, 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031160469999999998, 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.199945+0000", 2024-09-17T22:57:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.199945+0000", 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.199946+0000", 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2900000000000004e-07 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200020+0000", 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4194000000000001e-05 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200031+0000", 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0927999999999999e-05 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200036+0000", 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6800000000000001e-06 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200076+0000", 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9994999999999999e-05 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200090+0000", 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4331999999999999e-05 2024-09-17T22:57:21.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203031+0000", 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029413960000000002 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203061+0000", 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9992999999999999e-05 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.200536+0000", 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.498530858999999, 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029824729999999998, 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200536+0000", 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200536+0000", 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200537+0000", 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200543+0000", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200557+0000", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3444999999999999e-05 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200565+0000", 2024-09-17T22:57:21.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9750000000000007e-06 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200607+0000", 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2784999999999998e-05 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.200622+0000", 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4297999999999999e-05 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203483+0000", 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028613879999999999 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203519+0000", 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'16)", 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.201195+0000", 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.497871863, 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023426710000000002, 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201195+0000", 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201195+0000", 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201196+0000", 2024-09-17T22:57:21.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201211+0000", 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5206e-05 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201219+0000", 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5259999999999993e-06 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201224+0000", 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.481e-06 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201274+0000", 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0519999999999997e-05 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201289+0000", 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4601e-05 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203520+0000", 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022313049999999998 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203538+0000", 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7575e-05 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.717 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'16)", 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.201700+0000", 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.497366563, 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001855897, 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201700+0000", 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201700+0000", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201701+0000", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7300000000000005e-07 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201753+0000", 2024-09-17T22:57:21.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1666999999999998e-05 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201766+0000", 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3213e-05 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201771+0000", 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2340000000000002e-06 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201850+0000", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8592999999999998e-05 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.201866+0000", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5874000000000001e-05 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203540+0000", 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016737060000000001 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.203556+0000", 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6736999999999999e-05 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:85 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.251181+0000", 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.447886045000001, 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029413540000000002, 2024-09-17T22:57:21.722 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251181+0000", 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251181+0000", 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251181+0000", 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251189+0000", 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4569999999999999e-06 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251197+0000", 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4589999999999998e-06 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251202+0000", 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8590000000000004e-06 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251230+0000", 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8058000000000001e-05 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251242+0000", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1231e-05 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.254090+0000", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028485089999999999 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.254122+0000", 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2251e-05 2024-09-17T22:57:21.726 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.251770+0000", 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.447297239000001, 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053839830000000002, 2024-09-17T22:57:21.727 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251770+0000", 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251770+0000", 2024-09-17T22:57:21.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251770+0000", 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3799999999999998e-07 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251773+0000", 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5330000000000001e-06 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251782+0000", 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3999999999999998e-06 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251785+0000", 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.348e-06 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251820+0000", 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4638999999999998e-05 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.251834+0000", 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3871e-05 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257117+0000", 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005282714 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257154+0000", 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.714e-05 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.732 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'53)", 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.252379+0000", 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.446687594, 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047908109999999999, 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252379+0000", 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252379+0000", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252380+0000", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252409+0000", 2024-09-17T22:57:21.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8690000000000001e-05 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252420+0000", 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1323e-05 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252425+0000", 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9729999999999997e-06 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252462+0000", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7001e-05 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.252474+0000", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1864e-05 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257155+0000", 2024-09-17T22:57:21.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046815260000000001 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257170+0000", 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4722999999999999e-05 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:88 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'58, mlcod=16'53)", 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.253549+0000", 2024-09-17T22:57:21.737 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.445517796000001, 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0036350990000000001, 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253549+0000", 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253549+0000", 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253550+0000", 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1600000000000001e-07 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253555+0000", 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7600000000000002e-06 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253566+0000", 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1083999999999999e-05 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253571+0000", 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253616+0000", 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4965000000000001e-05 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.253630+0000", 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4751e-05 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257172+0000", 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0035414029999999998 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.257184+0000", 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2490999999999999e-05 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:90 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'60, mlcod=16'57)", 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.258520+0000", 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.440547309999999, 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0040217660000000004, 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258520+0000", 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258520+0000", 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258520+0000", 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5199999999999996e-07 2024-09-17T22:57:21.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258566+0000", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5908999999999997e-05 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258575+0000", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2970000000000004e-06 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258579+0000", 2024-09-17T22:57:21.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0879999999999997e-06 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258647+0000", 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8390000000000001e-05 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.258661+0000", 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3711e-05 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.262512+0000", 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0038508739999999998 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.262541+0000", 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9845e-05 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:116 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'78, mlcod=16'68)", 2024-09-17T22:57:21.747 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.377373+0000", 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.321694084000001, 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002825466, 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377373+0000", 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377373+0000", 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377373+0000", 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377379+0000", 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2460000000000003e-06 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377383+0000", 2024-09-17T22:57:21.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8790000000000002e-06 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377389+0000", 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2989999999999996e-06 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377419+0000", 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0185e-05 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.377430+0000", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1377e-05 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.380177+0000", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0027462879999999999 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.380198+0000", 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1732000000000001e-05 2024-09-17T22:57:21.752 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:117 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'76)", 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.378526+0000", 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.320540940000001, 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001852521, 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.753 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378526+0000", 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378526+0000", 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378527+0000", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378531+0000", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1309999999999999e-06 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378541+0000", 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7929999999999992e-06 2024-09-17T22:57:21.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378545+0000", 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6929999999999998e-06 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378580+0000", 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5162e-05 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.378591+0000", 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0837e-05 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.380361+0000", 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017694309999999999 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.380379+0000", 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7878999999999999e-05 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.757 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'89)", 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.781699+0000", 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.917368080999999, 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030666999999999999, 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781699+0000", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781699+0000", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781699+0000", 2024-09-17T22:57:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781705+0000", 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4770000000000002e-06 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781711+0000", 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.959e-06 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781714+0000", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6440000000000002e-06 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781744+0000", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9833999999999999e-05 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.781756+0000", 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1569000000000001e-05 2024-09-17T22:57:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.784736+0000", 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002980098 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.784766+0000", 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9672999999999999e-05 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'90)", 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.782133+0000", 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.916933792000002, 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030158810000000002, 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782133+0000", 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782133+0000", 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782134+0000", 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2300000000000002e-07 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782196+0000", 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2185000000000002e-05 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782203+0000", 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1149999999999996e-06 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782208+0000", 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1649999999999997e-06 2024-09-17T22:57:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782239+0000", 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1309999999999997e-05 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.782251+0000", 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.202e-05 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.785115+0000", 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028636550000000001 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.785149+0000", 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4007999999999997e-05 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'94)", 2024-09-17T22:57:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.786003+0000", 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.913063586, 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028879930000000002, 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786003+0000", 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786003+0000", 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786004+0000", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786009+0000", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9969999999999998e-06 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786025+0000", 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6033e-05 2024-09-17T22:57:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786032+0000", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786069+0000", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7209000000000002e-05 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.786082+0000", 2024-09-17T22:57:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.324e-05 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.788867+0000", 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002785114 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.788891+0000", 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4281e-05 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.790921+0000", 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.908146402, 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033856099999999998, 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.790921+0000", 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.790921+0000", 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.790921+0000", 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0900000000000001e-07 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.790993+0000", 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1664999999999999e-05 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791001+0000", 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791006+0000", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.383e-06 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791103+0000", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7145000000000006e-05 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791117+0000", 2024-09-17T22:57:21.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3271999999999999e-05 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794283+0000", 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031657970000000001 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794306+0000", 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3625000000000002e-05 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:144 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.791349+0000", 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.907718346999999, 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032377040000000001, 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.778 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791349+0000", 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791349+0000", 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791349+0000", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.13e-07 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791405+0000", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6143000000000001e-05 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791414+0000", 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0149999999999992e-06 2024-09-17T22:57:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791422+0000", 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6260000000000003e-06 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791453+0000", 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1112000000000003e-05 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791463+0000", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0352e-05 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794545+0000", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00308187 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794586+0000", 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0973000000000002e-05 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:145 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:04.791734+0000", 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.907332893, 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028831780000000001, 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791734+0000", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791734+0000", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791735+0000", 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000002e-07 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791800+0000", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5467000000000002e-05 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791807+0000", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0260000000000002e-06 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791811+0000", 2024-09-17T22:57:21.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.613e-06 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791836+0000", 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5327e-05 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.791847+0000", 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0764999999999999e-05 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794590+0000", 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0027435599999999999 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:04.794617+0000", 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6999e-05 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.787 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:05.242438+0000", 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.456628797, 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017428750000000001, 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242438+0000", 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242438+0000", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242439+0000", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242444+0000", 2024-09-17T22:57:21.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.626e-06 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242450+0000", 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2550000000000003e-06 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242454+0000", 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3850000000000002e-06 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242494+0000", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0175000000000001e-05 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.242511+0000", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7232999999999999e-05 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244147+0000", 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001635367 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244181+0000", 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4455000000000001e-05 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:156 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=16'105)", 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:05.243283+0000", 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.455783836999998, 2024-09-17T22:57:21.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0020142300000000001, 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243283+0000", 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243283+0000", 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243284+0000", 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243289+0000", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1410000000000004e-06 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243295+0000", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1500000000000004e-06 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243298+0000", 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2140000000000001e-06 2024-09-17T22:57:21.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243683+0000", 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038499399999999999 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.243703+0000", 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9425000000000001e-05 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.245270+0000", 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0015671400000000001 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.245297+0000", 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7725999999999998e-05 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.797 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4213.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'106)", 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:05.244099+0000", 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.454968359999999, 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001974178, 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244099+0000", 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244099+0000", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244099+0000", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244174+0000", 2024-09-17T22:57:21.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5556000000000006e-05 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244180+0000", 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1000000000000003e-06 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244183+0000", 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.033e-06 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244209+0000", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6021999999999999e-05 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.244221+0000", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1985e-05 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.246044+0000", 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018236979999999999 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:21.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:05.246073+0000", 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8483000000000002e-05 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:21.802 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:21.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:21.804 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:21.801+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:21.849 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:57:21.905 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:21.903+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:21.958 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:21.959 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:21.959 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:21.959 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:21.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:22.008 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:22.006+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:22.106 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:22.103+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:22.114 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:22.114 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:22.114 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:22.114 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:22.114 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:22.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:22.208 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:22.205+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:22.269 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:22.269 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:22.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:22.269 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 11.373804740000001, 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00080673799999999999, 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:22.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:57:22.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893419+0000", 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8020000000000003e-06 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893428+0000", 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2080000000000006e-06 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:22.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893437+0000", 2024-09-17T22:57:22.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3649999999999993e-06 2024-09-17T22:57:22.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893496+0000", 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8489999999999997e-05 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893523+0000", 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7055000000000001e-05 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.894177+0000", 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00065433799999999997 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.894215+0000", 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.82e-05 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.367103565000001, 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098273299999999996, 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.36e-07 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900118+0000", 2024-09-17T22:57:22.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0099999999999995e-06 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900155+0000", 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6925000000000003e-05 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900169+0000", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3723000000000001e-05 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900230+0000", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1554000000000001e-05 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900249+0000", 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8303000000000001e-05 2024-09-17T22:57:22.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.901037+0000", 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00078825800000000001 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.901093+0000", 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5723999999999997e-05 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7 3.1a e24/22 3:5cea7035:::rbd_info:head v 24'1, mlcod=0'0)", 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.4211513870000001, 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000673235, 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:22.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9000000000000001e-07 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846067+0000", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8910000000000003e-06 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846077+0000", 2024-09-17T22:57:22.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8120000000000002e-06 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846084+0000", 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3329999999999999e-06 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846127+0000", 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2845000000000001e-05 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:22.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846149+0000", 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2016999999999999e-05 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846697+0000", 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054781100000000004 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846735+0000", 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8336e-05 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.285 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.42104750200000002, 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017871269999999999, 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:22.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846170+0000", 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6770000000000001e-06 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846179+0000", 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9619999999999999e-06 2024-09-17T22:57:22.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846188+0000", 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1379999999999996e-06 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846241+0000", 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3625000000000002e-05 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:22.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846258+0000", 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6730000000000001e-05 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.847904+0000", 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016458569999999999 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.847953+0000", 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8908e-05 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.290 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.291 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.291 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.291 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:22.292 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:22.309 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:22.306+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:22.409 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:22.407+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:22.437 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:22.438 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:22.447 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:22.510 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:22.508+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:22.591 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:22.591 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:22.591 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:22.591 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:22.591 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:22.600 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:22.643 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:22.641+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:22.650 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:22.705 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:22.705 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:22.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:22.705 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:22.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 12.812475702, 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:22.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:22.710 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:22.711 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:22.709+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:22.714 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:22.774 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:22.774 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:22.791 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:22.791 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:22.808 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:22.808 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:22.812 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:22.810+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:22.824 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:22.841 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:22.842 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:22.859 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:22.859 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:22.860 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:22.861 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:22.870 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:22.876 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:22.877 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:22.895 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:22.896 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:22.913 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:22.911+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:22.984 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:22.984 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:22.993 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:23.036 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:23.034+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:23.098 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.098 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.107 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:23.153 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:23.151+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:23.216 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:23.214+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:23.224 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.224 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.234 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:23.317 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:23.315+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:23.379 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.379 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.388 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:23.432 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:23.429+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:23.492 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.492 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.501 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:23.542 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:23.540+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:23.565 INFO:teuthology.orchestra.run.smithi159.stdout:1 write 0x5186c6c thru 0x5194e24 (0xe1b9 bytes) 2024-09-17T22:57:23.568 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.568 INFO:teuthology.orchestra.run.smithi159.stdout:4 read 0x1422a4b thru 0x14328f4 (0xfeaa bytes) 2024-09-17T22:57:23.568 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.569 INFO:teuthology.orchestra.run.smithi159.stdout:7 writesame 0x1edc7c1 thru 0x1ee47c4 data_size 0x6(0x8004 bytes) 2024-09-17T22:57:23.572 INFO:teuthology.orchestra.run.smithi159.stdout:8 write 0x940cb6 thru 0x947a3d (0x6d88 bytes) 2024-09-17T22:57:23.577 INFO:teuthology.orchestra.run.smithi159.stdout:9 read 0x1878607 thru 0x187b8a0 (0x329a bytes) 2024-09-17T22:57:23.577 INFO:teuthology.orchestra.run.smithi159.stdout:11 read 0x373b9eb thru 0x3743954 (0x7f6a bytes) 2024-09-17T22:57:23.577 INFO:teuthology.orchestra.run.smithi159.stdout:12 write 0x4fb73c6 thru 0x4fbd80f (0x644a bytes) 2024-09-17T22:57:23.597 INFO:teuthology.orchestra.run.smithi159.stdout:13 writesame 0xd8dbe7f thru 0xd8dfebe data_size 0x20(0x4040 bytes) 2024-09-17T22:57:23.598 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.598 INFO:teuthology.orchestra.run.smithi159.stdout:17 read 0xa80a1eb thru 0xa80f477 (0x528d bytes) 2024-09-17T22:57:23.598 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.598 INFO:teuthology.orchestra.run.smithi159.stdout:19 write 0xa1cac6d thru 0xa1d3ec5 (0x9259 bytes) 2024-09-17T22:57:23.601 INFO:teuthology.orchestra.run.smithi159.stdout:22 write 0x8f027e6 thru 0x8f0edfd (0xc618 bytes) 2024-09-17T22:57:23.605 INFO:teuthology.orchestra.run.smithi159.stdout:24 read 0xa825ed8 thru 0xa82d909 (0x7a32 bytes) 2024-09-17T22:57:23.605 INFO:teuthology.orchestra.run.smithi159.stdout:25 write 0x4644477 thru 0x46486d5 (0x425f bytes) 2024-09-17T22:57:23.607 INFO:teuthology.orchestra.run.smithi159.stdout:26 write 0x5e945f thru 0x5ecc65 (0x3807 bytes) 2024-09-17T22:57:23.610 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.610 INFO:teuthology.orchestra.run.smithi159.stdout:29 read 0xc3217b2 thru 0xc328ee1 (0x7730 bytes) 2024-09-17T22:57:23.610 INFO:teuthology.orchestra.run.smithi159.stdout:31 writesame 0x845410b thru 0x84610fe data_size 0x2(0xcff4 bytes) 2024-09-17T22:57:23.619 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:23.617+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:23.626 INFO:teuthology.orchestra.run.smithi159.stdout:32 read 0xac43de3 thru 0xac451dd (0x13fb bytes) 2024-09-17T22:57:23.626 INFO:teuthology.orchestra.run.smithi159.stdout:33 writesame 0xca07052 thru 0xca0a1c2 data_size 0x1(0x3171 bytes) 2024-09-17T22:57:23.636 INFO:teuthology.orchestra.run.smithi159.stdout:34 read 0x9f9c84e thru 0x9fa7e82 (0xb635 bytes) 2024-09-17T22:57:23.636 INFO:teuthology.orchestra.run.smithi159.stdout:35 write 0x644af58 thru 0x64582c9 (0xd372 bytes) 2024-09-17T22:57:23.639 INFO:teuthology.orchestra.run.smithi159.stdout:36 punch from 0xd66ead0 to 0xd67a90c, (0xbe3c bytes) 2024-09-17T22:57:23.639 INFO:teuthology.orchestra.run.smithi159.stdout:37 writesame 0x1ada516 thru 0x1ae4f51 data_size 0xa(0xaa3c bytes) 2024-09-17T22:57:23.644 INFO:teuthology.orchestra.run.smithi159.stdout:38 writesame 0xa032643 thru 0xa03651f data_size 0x13(0x3edd bytes) 2024-09-17T22:57:23.646 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.646 INFO:teuthology.orchestra.run.smithi159.stdout:40 write 0x7bba3fd thru 0x7bbff20 (0x5b24 bytes) 2024-09-17T22:57:23.649 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.649 INFO:teuthology.orchestra.run.smithi159.stdout:42 punch from 0xa149af to 0xa1901a, (0x466b bytes) 2024-09-17T22:57:23.649 INFO:teuthology.orchestra.run.smithi159.stdout:43 writesame 0x81439e2 thru 0x815013f data_size 0x2(0xc75e bytes) 2024-09-17T22:57:23.651 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.652 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.661 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:23.668 INFO:teuthology.orchestra.run.smithi159.stdout:44 write 0xe12d690 thru 0xe131caf (0x4620 bytes) 2024-09-17T22:57:23.672 INFO:teuthology.orchestra.run.smithi159.stdout:45 punch from 0xd5353d5 to 0xd5422fd, (0xcf28 bytes) 2024-09-17T22:57:23.672 INFO:teuthology.orchestra.run.smithi159.stdout:47 write 0x2e28646 thru 0x2e2f2eb (0x6ca6 bytes) 2024-09-17T22:57:23.675 INFO:teuthology.orchestra.run.smithi159.stdout:48 punch from 0xc363702 to 0xc3672e0, (0x3bde bytes) 2024-09-17T22:57:23.676 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0x30db9ac 2024-09-17T22:57:23.688 INFO:teuthology.orchestra.run.smithi159.stdout:49 trunc from 0xe131cb0 to 0x30db9ac 2024-09-17T22:57:23.703 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:23.703 INFO:teuthology.orchestra.run.smithi159.stdout:54 read 0x279c0ea thru 0x27a2a18 (0x692f bytes) 2024-09-17T22:57:23.706 INFO:teuthology.orchestra.run.smithi159.stdout:55 trunc from 0x30db9ac to 0x4e88d0 2024-09-17T22:57:23.717 INFO:teuthology.orchestra.run.smithi159.stdout:56 read 0x444aef thru 0x44fcf0 (0xb202 bytes) 2024-09-17T22:57:23.720 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:23.718+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:23.731 INFO:teuthology.orchestra.run.smithi159.stdout:58 write 0x71d0877 thru 0x71de5f9 (0xdd83 bytes) 2024-09-17T22:57:23.734 INFO:teuthology.orchestra.run.smithi159.stdout:60 write 0x16eca40 thru 0x16fb15c (0xe71d bytes) 2024-09-17T22:57:23.737 INFO:teuthology.orchestra.run.smithi159.stdout:61 writesame 0x57ea8ad thru 0x57ec6db data_size 0x1(0x1e2f bytes) 2024-09-17T22:57:23.741 INFO:teuthology.orchestra.run.smithi159.stdout:62 read 0x47da729 thru 0x47e9d19 (0xf5f1 bytes) 2024-09-17T22:57:23.742 INFO:teuthology.orchestra.run.smithi159.stdout:63 clone 1 order 25 su 131072 sc 3 2024-09-17T22:57:23.768 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.768 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.777 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:23.821 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:23.819+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:23.865 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:57:23.880 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:23.880 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:23.889 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:23.922 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:23.920+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:24.024 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:24.021+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:24.035 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.035 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:24.079 INFO:teuthology.orchestra.run.smithi159.stdout:64 read 0xffe75b thru 0x1005ef1 (0x7797 bytes) 2024-09-17T22:57:24.090 INFO:teuthology.orchestra.run.smithi159.stdout:65 write 0x886d447 thru 0x88724b8 (0x5072 bytes) 2024-09-17T22:57:24.095 INFO:teuthology.orchestra.run.smithi159.stdout:66 read 0x70178f0 thru 0x7021802 (0x9f13 bytes) 2024-09-17T22:57:24.096 INFO:teuthology.orchestra.run.smithi159.stdout:67 read 0x1efc63c thru 0x1f05af6 (0x94bb bytes) 2024-09-17T22:57:24.096 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.096 INFO:teuthology.orchestra.run.smithi159.stdout:69 read 0x1fca918 thru 0x1fda592 (0xfc7b bytes) 2024-09-17T22:57:24.096 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0x731ad7a 2024-09-17T22:57:24.100 INFO:teuthology.orchestra.run.smithi159.stdout:70 trunc from 0x88724b9 to 0x731ad7a 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:72 read 0x4f5db6e thru 0x4f6c7e2 (0xec75 bytes) 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:74 punch from 0x4a50661 to 0x4a55b59, (0x54f8 bytes) 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.113 INFO:teuthology.orchestra.run.smithi159.stdout:77 write 0x579500c thru 0x579c398 (0x738d bytes) 2024-09-17T22:57:24.120 INFO:teuthology.orchestra.run.smithi159.stdout:78 read 0x5b6ac53 thru 0x5b75f06 (0xb2b4 bytes) 2024-09-17T22:57:24.120 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.120 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0x7a80213 2024-09-17T22:57:24.120 INFO:teuthology.orchestra.run.smithi159.stdout:80 trunc from 0x731ad7a to 0x7a80213 2024-09-17T22:57:24.124 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:24.122+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:24.125 INFO:teuthology.orchestra.run.smithi159.stdout:81 write 0x4162a6c thru 0x416e09c (0xb631 bytes) 2024-09-17T22:57:24.126 INFO:teuthology.orchestra.run.smithi159.stdout:82 write 0x74ceb6a thru 0x74ddcb6 (0xf14d bytes) 2024-09-17T22:57:24.128 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.128 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.128 INFO:teuthology.orchestra.run.smithi159.stdout:85 punch from 0x7a06558 to 0x7a14027, (0xdacf bytes) 2024-09-17T22:57:24.128 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.141 INFO:teuthology.orchestra.run.smithi159.stdout:87 writesame 0xdf8686f thru 0xdf91b22 data_size 0x2(0xb2b4 bytes) 2024-09-17T22:57:24.141 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.149 INFO:teuthology.orchestra.run.smithi159.stdout:92 trunc from 0xdf91b23 to 0x72e614c 2024-09-17T22:57:24.153 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.153 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.162 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:24.167 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:24.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:24.167 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T22:57:24.168 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:57:24.170 INFO:teuthology.orchestra.run.smithi159.stdout:93 write 0xdf95725 thru 0xdf99a1a (0x42f6 bytes) 2024-09-17T22:57:24.186 INFO:teuthology.orchestra.run.smithi159.stdout:94 trunc from 0xdf99a1b to 0x329e570 2024-09-17T22:57:24.204 INFO:teuthology.orchestra.run.smithi159.stdout:96 read 0xa25bcc thru 0xa28fa4 (0x33d9 bytes) 2024-09-17T22:57:24.205 INFO:teuthology.orchestra.run.smithi159.stdout:97 punch from 0x14c3d86 to 0x14d0e2a, (0xd0a4 bytes) 2024-09-17T22:57:24.219 INFO:teuthology.orchestra.run.smithi159.stdout:98 write 0x8129b14 thru 0x81343ff (0xa8ec bytes) 2024-09-17T22:57:24.225 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:24.223+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:24.232 INFO:teuthology.orchestra.run.smithi159.stdout:99 write 0xb06c11a thru 0xb077493 (0xb37a bytes) 2024-09-17T22:57:24.233 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.233 INFO:teuthology.orchestra.run.smithi159.stdout:102 read 0x23d1cb1 thru 0x23e02ae (0xe5fe bytes) 2024-09-17T22:57:24.233 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0xe14f8a6 2024-09-17T22:57:24.237 INFO:teuthology.orchestra.run.smithi159.stdout:103 trunc from 0xb077494 to 0xe14f8a6 2024-09-17T22:57:24.243 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.243 INFO:teuthology.orchestra.run.smithi159.stdout:105 write 0x54f0c5d thru 0x54faa29 (0x9dcd bytes) 2024-09-17T22:57:24.244 INFO:teuthology.orchestra.run.smithi159.stdout:106 write 0x9cb55aa thru 0x9cbe2b5 (0x8d0c bytes) 2024-09-17T22:57:24.247 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.251 INFO:teuthology.orchestra.run.smithi159.stdout:108 trunc from 0xe14f8a6 to 0xbe2f67e 2024-09-17T22:57:24.257 INFO:teuthology.orchestra.run.smithi159.stdout:109 read 0x5fd583b thru 0x5fdac0f (0x53d5 bytes) 2024-09-17T22:57:24.260 INFO:teuthology.orchestra.run.smithi159.stdout:114 trunc from 0xbe2f67e to 0x992211b 2024-09-17T22:57:24.268 INFO:teuthology.orchestra.run.smithi159.stdout:115 punch from 0x3728cd8 to 0x372fd1b, (0x7043 bytes) 2024-09-17T22:57:24.268 INFO:teuthology.orchestra.run.smithi159.stdout:116 write 0x55adcf thru 0x569087 (0xe2b9 bytes) 2024-09-17T22:57:24.270 INFO:teuthology.orchestra.run.smithi159.stdout:117 writesame 0x15c30b1 thru 0x15d13e3 data_size 0x7(0xe333 bytes) 2024-09-17T22:57:24.274 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:24.274 INFO:teuthology.orchestra.run.smithi159.stdout:119 write 0x7c817c3 thru 0x7c8d32b (0xbb69 bytes) 2024-09-17T22:57:24.274 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.274 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.278 INFO:teuthology.orchestra.run.smithi159.stdout:121 write 0x771a0e1 thru 0x771a536 (0x456 bytes) 2024-09-17T22:57:24.285 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:24.292 INFO:teuthology.orchestra.run.smithi159.stdout:122 write 0xe98c4da thru 0xe990eb0 (0x49d7 bytes) 2024-09-17T22:57:24.296 INFO:teuthology.orchestra.run.smithi159.stdout:124 write 0x50a81ff thru 0x50b47aa (0xc5ac bytes) 2024-09-17T22:57:24.297 INFO:teuthology.orchestra.run.smithi159.stdout:125 clone 2 order 18 su 131072 sc 2 2024-09-17T22:57:24.328 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:24.326+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:24.388 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.388 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.397 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:24.438 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:24.436+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: "epoch": 26, 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: "fsid": "33081d94-d15e-4bd4-ba4c-30022eb1be7a", 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: "created": "2024-09-17T22:56:50.056303+0000", 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: "modified": "2024-09-17T22:57:23.944305+0000", 2024-09-17T22:57:24.474 INFO:teuthology.orchestra.run.smithi159.stdout: "last_up_change": "2024-09-17T22:57:02.843283+0000", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "last_in_change": "2024-09-17T22:56:53.064036+0000", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_num": 5806080, 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_set": [ 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "nodeep-scrub", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "noscrub", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "pglog_hardlimit", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "purged_snapdirs", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_deletes", 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "sortbitwise" 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_version": 5, 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:57:24.475 INFO:teuthology.orchestra.run.smithi159.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_snapshot": "", 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_max": 3, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd": 8, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "min_compat_client": "jewel", 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "require_osd_release": "squid", 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_crimson": false, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "pools": [ 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 1, 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": ".mgr", 2024-09-17T22:57:24.476 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:02.221581+0000", 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool", 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:24.477 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 1, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 1, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 1, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 1, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 1, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:24.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "17", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 0, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 0, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:24.479 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:24.480 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:24.481 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "options": { 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_max": 32, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_min": 1 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": {} 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 8, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 8, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.25, 2024-09-17T22:57:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 2, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "rbd", 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:08.859324+0000", 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:57:24.483 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 8, 2024-09-17T22:57:24.484 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 8, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 8, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 8, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 8, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "21", 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:24.485 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 2, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 21, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:24.486 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:24.487 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:24.488 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.9900000095367432, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.9900000095367432, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.75, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.75, 2024-09-17T22:57:24.489 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "pool_client.0", 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:17.992936+0000", 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:24.490 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 32, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 32, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 32, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 32, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 32, 2024-09-17T22:57:24.491 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "26", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:24.492 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 3, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 26, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:24.493 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:24.494 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.5, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.5, 2024-09-17T22:57:24.495 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 1, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.5, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.5, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "892321c3-409f-40e6-bfe1-c87f464d030f", 2024-09-17T22:57:24.496 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.497 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6808", 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6809", 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.498 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6810", 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6811", 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.499 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6814", 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6815", 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6812", 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6813", 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6809/2087020665", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6811/2087020665", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6815/2087020665", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6813/2087020665", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "cd777f15-a6ad-4f0c-820b-ef529dd9f008", 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.502 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.503 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6800", 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6801", 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.504 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6802", 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6803", 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.505 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6806", 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6807", 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.509 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6804", 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6805", 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:24.510 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6801/237398697", 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6803/237398697", 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6807/237398697", 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6805/237398697", 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:57:24.511 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "bd68e905-6e66-4735-9de8-66a99530420f", 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 18, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6816", 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6817", 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6818", 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6819", 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6822", 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6823", 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.515 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6820", 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6821", 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.516 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6817/927278608", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6819/927278608", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6823/927278608", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6821/927278608", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "28b3b7ca-d7d7-4ed9-8dfe-113d3579209c", 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.517 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.518 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6824", 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6825", 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.519 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6826", 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6827", 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6830", 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6831", 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6828", 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6829", 2024-09-17T22:57:24.522 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6825/2987856795", 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6827/2987856795", 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6831/2987856795", 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6829/2987856795", 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.523 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "d379c252-e66a-4d0e-af0a-81f6bda48c08", 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.524 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6826", 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6827", 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.525 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6828", 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6829", 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.526 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6832", 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6833", 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.527 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6830", 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6831", 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6827/2123531187", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6829/2123531187", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6833/2123531187", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6831/2123531187", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "e78a100f-1ff3-4af6-abd8-a069030ec951", 2024-09-17T22:57:24.529 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.530 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6818", 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6819", 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.531 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6820", 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6821", 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.532 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6824", 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6825", 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.533 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6822", 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6823", 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6819/3064143709", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6821/3064143709", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6825/3064143709", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6823/3064143709", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "fc1af848-dfaa-4e40-a0af-8982c70f7ea9", 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.535 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.536 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6810", 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6811", 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6812", 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6813", 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.538 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6816", 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6817", 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.539 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6814", 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6815", 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6811/1505991260", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6813/1505991260", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6817/1505991260", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6815/1505991260", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "ccd75a18-ded4-4f75-a09a-926da5362a7f", 2024-09-17T22:57:24.541 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 13, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.542 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6800", 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6801", 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.543 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6802", 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6803", 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.544 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6806", 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6807", 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:24.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6804", 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6805", 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6801/585405482", 2024-09-17T22:57:24.546 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6803/585405482", 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6807/585405482", 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6805/585405482", 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_xinfo": [ 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.547 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.101775+0000", 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.929708+0000", 2024-09-17T22:57:24.548 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.562720+0000", 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.549 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.569375+0000", 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.550 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:57:00.002784+0000", 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.775667+0000", 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.558418+0000", 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:24.552 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.840790+0000", 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap": [], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_items": [], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_primaries": [], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_temp": [], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_temp": [], 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "blocklist": {}, 2024-09-17T22:57:24.553 INFO:teuthology.orchestra.run.smithi159.stdout: "range_blocklist": {}, 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profiles": { 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "default": { 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-failure-domain": "osd", 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "k": "2", 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "m": "1", 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin": "jerasure", 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "technique": "reed_sol_van" 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps_queue": [ 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:57:24.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 2, 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 1 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "new_removed_snaps": [], 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "new_purged_snaps": [], 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_node_flags": {}, 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "device_class_flags": {}, 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode": { 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_enabled": false, 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_bucket_count": 0, 2024-09-17T22:57:24.555 INFO:teuthology.orchestra.run.smithi159.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:57:24.556 INFO:teuthology.orchestra.run.smithi159.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:57:24.556 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_bucket": 0 2024-09-17T22:57:24.556 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:24.556 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:24.558 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.558 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.559 INFO:tasks.thrashosds.thrasher:j is {'epoch': 26, 'fsid': '33081d94-d15e-4bd4-ba4c-30022eb1be7a', 'created': '2024-09-17T22:56:50.056303+0000', 'modified': '2024-09-17T22:57:23.944305+0000', 'last_up_change': '2024-09-17T22:57:02.843283+0000', 'last_in_change': '2024-09-17T22:56:53.064036+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:57:02.221581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:57:08.859324+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:57:17.992936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 26, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '892321c3-409f-40e6-bfe1-c87f464d030f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6808', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6809', 'nonce': 2087020665}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6810', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6811', 'nonce': 2087020665}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6814', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6815', 'nonce': 2087020665}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6812', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6813', 'nonce': 2087020665}]}, 'public_addr': '172.21.15.159:6809/2087020665', 'cluster_addr': '172.21.15.159:6811/2087020665', 'heartbeat_back_addr': '172.21.15.159:6815/2087020665', 'heartbeat_front_addr': '172.21.15.159:6813/2087020665', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'cd777f15-a6ad-4f0c-820b-ef529dd9f008', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6800', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6801', 'nonce': 237398697}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6802', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6803', 'nonce': 237398697}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6806', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6807', 'nonce': 237398697}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6804', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6805', 'nonce': 237398697}]}, 'public_addr': '172.21.15.159:6801/237398697', 'cluster_addr': '172.21.15.159:6803/237398697', 'heartbeat_back_addr': '172.21.15.159:6807/237398697', 'heartbeat_front_addr': '172.21.15.159:6805/237398697', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bd68e905-6e66-4735-9de8-66a99530420f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6816', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6817', 'nonce': 927278608}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6818', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6819', 'nonce': 927278608}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6822', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6823', 'nonce': 927278608}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6820', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6821', 'nonce': 927278608}]}, 'public_addr': '172.21.15.159:6817/927278608', 'cluster_addr': '172.21.15.159:6819/927278608', 'heartbeat_back_addr': '172.21.15.159:6823/927278608', 'heartbeat_front_addr': '172.21.15.159:6821/927278608', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '28b3b7ca-d7d7-4ed9-8dfe-113d3579209c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6824', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6825', 'nonce': 2987856795}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6826', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6827', 'nonce': 2987856795}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6830', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6831', 'nonce': 2987856795}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6828', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6829', 'nonce': 2987856795}]}, 'public_addr': '172.21.15.159:6825/2987856795', 'cluster_addr': '172.21.15.159:6827/2987856795', 'heartbeat_back_addr': '172.21.15.159:6831/2987856795', 'heartbeat_front_addr': '172.21.15.159:6829/2987856795', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd379c252-e66a-4d0e-af0a-81f6bda48c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2123531187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2123531187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2123531187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2123531187}]}, 'public_addr': '172.21.15.171:6827/2123531187', 'cluster_addr': '172.21.15.171:6829/2123531187', 'heartbeat_back_addr': '172.21.15.171:6833/2123531187', 'heartbeat_front_addr': '172.21.15.171:6831/2123531187', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e78a100f-1ff3-4af6-abd8-a069030ec951', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 3064143709}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 3064143709}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 3064143709}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 3064143709}]}, 'public_addr': '172.21.15.171:6819/3064143709', 'cluster_addr': '172.21.15.171:6821/3064143709', 'heartbeat_back_addr': '172.21.15.171:6825/3064143709', 'heartbeat_front_addr': '172.21.15.171:6823/3064143709', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'fc1af848-dfaa-4e40-a0af-8982c70f7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 1505991260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 1505991260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1505991260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 1505991260}]}, 'public_addr': '172.21.15.171:6811/1505991260', 'cluster_addr': '172.21.15.171:6813/1505991260', 'heartbeat_back_addr': '172.21.15.171:6817/1505991260', 'heartbeat_front_addr': '172.21.15.171:6815/1505991260', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ccd75a18-ded4-4f75-a09a-926da5362a7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 585405482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 585405482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 585405482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 585405482}]}, 'public_addr': '172.21.15.171:6801/585405482', 'cluster_addr': '172.21.15.171:6803/585405482', 'heartbeat_back_addr': '172.21.15.171:6807/585405482', 'heartbeat_front_addr': '172.21.15.171:6805/585405482', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.101775+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.929708+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.562720+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.569375+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:57:00.002784+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.775667+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.558418+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.840790+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:57:24.559 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:24.559 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:24.560 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:24.555+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:24.657 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:24.655+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:24.701 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.701 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.710 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:24.758 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:24.756+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:24.815 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.815 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.824 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:24.867 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:24.865+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:24.882 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:24.883 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:24.894 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-17T22:57:24.195268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":150,"num_read_kb":126,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223540,"kb_used_data":3132,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751180,"statfs":{"total":773094113280,"available":772865208320,"internally_reserved":0,"allocated":3207168,"data_stored":1912049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.354243"},"pg_stats":[{"pgid":"3.1f","version":"25'3","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.873557+0000","last_change":"2024-09-17T22:57:21.855741+0000","last_active":"2024-09-17T22:57:21.873557+0000","last_peered":"2024-09-17T22:57:21.873557+0000","last_clean":"2024-09-17T22:57:21.873557+0000","last_became_active":"2024-09-17T22:57:19.849385+0000","last_became_peered":"2024-09-17T22:57:19.849385+0000","last_unstale":"2024-09-17T22:57:21.873557+0000","last_undegraded":"2024-09-17T22:57:21.873557+0000","last_fullsized":"2024-09-17T22:57:21.873557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:41:54.351028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001658269,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.844862+0000","last_change":"2024-09-17T22:57:21.844862+0000","last_active":"2024-09-17T22:57:21.844862+0000","last_peered":"2024-09-17T22:57:21.844862+0000","last_clean":"2024-09-17T22:57:21.844862+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:57:21.844862+0000","last_undegraded":"2024-09-17T22:57:21.844862+0000","last_fullsized":"2024-09-17T22:57:21.844862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:57:31.517123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042388700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.851512+0000","last_change":"2024-09-17T22:57:21.851512+0000","last_active":"2024-09-17T22:57:21.851512+0000","last_peered":"2024-09-17T22:57:21.851512+0000","last_clean":"2024-09-17T22:57:21.851512+0000","last_became_active":"2024-09-17T22:57:19.853868+0000","last_became_peered":"2024-09-17T22:57:19.853868+0000","last_unstale":"2024-09-17T22:57:21.851512+0000","last_undegraded":"2024-09-17T22:57:21.851512+0000","last_fullsized":"2024-09-17T22:57:21.851512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:53:07.349763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010535799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"25'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.851670+0000","last_change":"2024-09-17T22:57:21.851236+0000","last_active":"2024-09-17T22:57:21.851670+0000","last_peered":"2024-09-17T22:57:21.851670+0000","last_clean":"2024-09-17T22:57:21.851670+0000","last_became_active":"2024-09-17T22:57:19.852990+0000","last_became_peered":"2024-09-17T22:57:19.852990+0000","last_unstale":"2024-09-17T22:57:21.851670+0000","last_undegraded":"2024-09-17T22:57:21.851670+0000","last_fullsized":"2024-09-17T22:57:21.851670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:28:18.368915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089476699999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.855970+0000","last_change":"2024-09-17T22:57:21.855970+0000","last_active":"2024-09-17T22:57:21.855970+0000","last_peered":"2024-09-17T22:57:21.855970+0000","last_clean":"2024-09-17T22:57:21.855970+0000","last_became_active":"2024-09-17T22:57:19.852369+0000","last_became_peered":"2024-09-17T22:57:19.852369+0000","last_unstale":"2024-09-17T22:57:21.855970+0000","last_undegraded":"2024-09-17T22:57:21.855970+0000","last_fullsized":"2024-09-17T22:57:21.855970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:16.548841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"25'2","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.872000+0000","last_change":"2024-09-17T22:57:21.845702+0000","last_active":"2024-09-17T22:57:21.872000+0000","last_peered":"2024-09-17T22:57:21.872000+0000","last_clean":"2024-09-17T22:57:21.872000+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:57:21.872000+0000","last_undegraded":"2024-09-17T22:57:21.872000+0000","last_fullsized":"2024-09-17T22:57:21.872000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:57:45.071461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.854734+0000","last_change":"2024-09-17T22:57:21.854734+0000","last_active":"2024-09-17T22:57:21.854734+0000","last_peered":"2024-09-17T22:57:21.854734+0000","last_clean":"2024-09-17T22:57:21.854734+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:57:21.854734+0000","last_undegraded":"2024-09-17T22:57:21.854734+0000","last_fullsized":"2024-09-17T22:57:21.854734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:03.725401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083302300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"25'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.864866+0000","last_change":"2024-09-17T22:57:21.845501+0000","last_active":"2024-09-17T22:57:21.864866+0000","last_peered":"2024-09-17T22:57:21.864866+0000","last_clean":"2024-09-17T22:57:21.864866+0000","last_became_active":"2024-09-17T22:57:19.848501+0000","last_became_peered":"2024-09-17T22:57:19.848501+0000","last_unstale":"2024-09-17T22:57:21.864866+0000","last_undegraded":"2024-09-17T22:57:21.864866+0000","last_fullsized":"2024-09-17T22:57:21.864866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:51.110616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090556399999999998,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.855764+0000","last_change":"2024-09-17T22:57:21.855764+0000","last_active":"2024-09-17T22:57:21.855764+0000","last_peered":"2024-09-17T22:57:21.855764+0000","last_clean":"2024-09-17T22:57:21.855764+0000","last_became_active":"2024-09-17T22:57:19.849370+0000","last_became_peered":"2024-09-17T22:57:19.849370+0000","last_unstale":"2024-09-17T22:57:21.855764+0000","last_undegraded":"2024-09-17T22:57:21.855764+0000","last_fullsized":"2024-09-17T22:57:21.855764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:07.171787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.851549+0000","last_change":"2024-09-17T22:57:21.851549+0000","last_active":"2024-09-17T22:57:21.851549+0000","last_peered":"2024-09-17T22:57:21.851549+0000","last_clean":"2024-09-17T22:57:21.851549+0000","last_became_active":"2024-09-17T22:57:19.850912+0000","last_became_peered":"2024-09-17T22:57:19.850912+0000","last_unstale":"2024-09-17T22:57:21.851549+0000","last_undegraded":"2024-09-17T22:57:21.851549+0000","last_fullsized":"2024-09-17T22:57:21.851549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:55.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092406,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843564+0000","last_change":"2024-09-17T22:57:19.845628+0000","last_active":"2024-09-17T22:57:20.843564+0000","last_peered":"2024-09-17T22:57:20.843564+0000","last_clean":"2024-09-17T22:57:20.843564+0000","last_became_active":"2024-09-17T22:57:19.845355+0000","last_became_peered":"2024-09-17T22:57:19.845355+0000","last_unstale":"2024-09-17T22:57:20.843564+0000","last_undegraded":"2024-09-17T22:57:20.843564+0000","last_fullsized":"2024-09-17T22:57:20.843564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:27:20.504206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.846023+0000","last_change":"2024-09-17T22:57:21.846023+0000","last_active":"2024-09-17T22:57:21.846023+0000","last_peered":"2024-09-17T22:57:21.846023+0000","last_clean":"2024-09-17T22:57:21.846023+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:57:21.846023+0000","last_undegraded":"2024-09-17T22:57:21.846023+0000","last_fullsized":"2024-09-17T22:57:21.846023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:16.806980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055329399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:22.374021+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:22.374021+0000","last_peered":"2024-09-17T22:57:22.374021+0000","last_clean":"2024-09-17T22:57:22.374021+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:22.374021+0000","last_undegraded":"2024-09-17T22:57:22.374021+0000","last_fullsized":"2024-09-17T22:57:22.374021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.854577+0000","last_change":"2024-09-17T22:57:21.854577+0000","last_active":"2024-09-17T22:57:21.854577+0000","last_peered":"2024-09-17T22:57:21.854577+0000","last_clean":"2024-09-17T22:57:21.854577+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:57:21.854577+0000","last_undegraded":"2024-09-17T22:57:21.854577+0000","last_fullsized":"2024-09-17T22:57:21.854577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:53:13.650966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053521000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.853737+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:21.853737+0000","last_peered":"2024-09-17T22:57:21.853737+0000","last_clean":"2024-09-17T22:57:21.853737+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:21.853737+0000","last_undegraded":"2024-09-17T22:57:21.853737+0000","last_fullsized":"2024-09-17T22:57:21.853737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845376+0000","last_change":"2024-09-17T22:57:21.845376+0000","last_active":"2024-09-17T22:57:21.845376+0000","last_peered":"2024-09-17T22:57:21.845376+0000","last_clean":"2024-09-17T22:57:21.845376+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:57:21.845376+0000","last_undegraded":"2024-09-17T22:57:21.845376+0000","last_fullsized":"2024-09-17T22:57:21.845376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:01.787339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000836865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843580+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:20.843580+0000","last_peered":"2024-09-17T22:57:20.843580+0000","last_clean":"2024-09-17T22:57:20.843580+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:20.843580+0000","last_undegraded":"2024-09-17T22:57:20.843580+0000","last_fullsized":"2024-09-17T22:57:20.843580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.854702+0000","last_change":"2024-09-17T22:57:21.854702+0000","last_active":"2024-09-17T22:57:21.854702+0000","last_peered":"2024-09-17T22:57:21.854702+0000","last_clean":"2024-09-17T22:57:21.854702+0000","last_became_active":"2024-09-17T22:57:19.951388+0000","last_became_peered":"2024-09-17T22:57:19.951388+0000","last_unstale":"2024-09-17T22:57:21.854702+0000","last_undegraded":"2024-09-17T22:57:21.854702+0000","last_fullsized":"2024-09-17T22:57:21.854702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:29:25.337454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080280599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.853956+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:21.853956+0000","last_peered":"2024-09-17T22:57:21.853956+0000","last_clean":"2024-09-17T22:57:21.853956+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:21.853956+0000","last_undegraded":"2024-09-17T22:57:21.853956+0000","last_fullsized":"2024-09-17T22:57:21.853956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.851751+0000","last_change":"2024-09-17T22:57:21.851751+0000","last_active":"2024-09-17T22:57:21.851751+0000","last_peered":"2024-09-17T22:57:21.851751+0000","last_clean":"2024-09-17T22:57:21.851751+0000","last_became_active":"2024-09-17T22:57:19.853015+0000","last_became_peered":"2024-09-17T22:57:19.853015+0000","last_unstale":"2024-09-17T22:57:21.851751+0000","last_undegraded":"2024-09-17T22:57:21.851751+0000","last_fullsized":"2024-09-17T22:57:21.851751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:17.040885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845990+0000","last_change":"2024-09-17T22:57:21.845990+0000","last_active":"2024-09-17T22:57:21.845990+0000","last_peered":"2024-09-17T22:57:21.845990+0000","last_clean":"2024-09-17T22:57:21.845990+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:57:21.845990+0000","last_undegraded":"2024-09-17T22:57:21.845990+0000","last_fullsized":"2024-09-17T22:57:21.845990+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:52.334191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058788700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.850324+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:21.850324+0000","last_peered":"2024-09-17T22:57:21.850324+0000","last_clean":"2024-09-17T22:57:21.850324+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:21.850324+0000","last_undegraded":"2024-09-17T22:57:21.850324+0000","last_fullsized":"2024-09-17T22:57:21.850324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.854973+0000","last_change":"2024-09-17T22:57:21.854973+0000","last_active":"2024-09-17T22:57:21.854973+0000","last_peered":"2024-09-17T22:57:21.854973+0000","last_clean":"2024-09-17T22:57:21.854973+0000","last_became_active":"2024-09-17T22:57:19.951410+0000","last_became_peered":"2024-09-17T22:57:19.951410+0000","last_unstale":"2024-09-17T22:57:21.854973+0000","last_undegraded":"2024-09-17T22:57:21.854973+0000","last_fullsized":"2024-09-17T22:57:21.854973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:58.001623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841561,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.844292+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:21.844292+0000","last_peered":"2024-09-17T22:57:21.844292+0000","last_clean":"2024-09-17T22:57:21.844292+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:21.844292+0000","last_undegraded":"2024-09-17T22:57:21.844292+0000","last_fullsized":"2024-09-17T22:57:21.844292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.854980+0000","last_change":"2024-09-17T22:57:21.854980+0000","last_active":"2024-09-17T22:57:21.854980+0000","last_peered":"2024-09-17T22:57:21.854980+0000","last_clean":"2024-09-17T22:57:21.854980+0000","last_became_active":"2024-09-17T22:57:19.852417+0000","last_became_peered":"2024-09-17T22:57:19.852417+0000","last_unstale":"2024-09-17T22:57:21.854980+0000","last_undegraded":"2024-09-17T22:57:21.854980+0000","last_fullsized":"2024-09-17T22:57:21.854980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:24.731735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081202100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843345+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:20.843345+0000","last_peered":"2024-09-17T22:57:20.843345+0000","last_clean":"2024-09-17T22:57:20.843345+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:20.843345+0000","last_undegraded":"2024-09-17T22:57:20.843345+0000","last_fullsized":"2024-09-17T22:57:20.843345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845112+0000","last_change":"2024-09-17T22:57:21.845112+0000","last_active":"2024-09-17T22:57:21.845112+0000","last_peered":"2024-09-17T22:57:21.845112+0000","last_clean":"2024-09-17T22:57:21.845112+0000","last_became_active":"2024-09-17T22:57:19.849300+0000","last_became_peered":"2024-09-17T22:57:19.849300+0000","last_unstale":"2024-09-17T22:57:21.845112+0000","last_undegraded":"2024-09-17T22:57:21.845112+0000","last_fullsized":"2024-09-17T22:57:21.845112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:28:32.643717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057890600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.850258+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:21.850258+0000","last_peered":"2024-09-17T22:57:21.850258+0000","last_clean":"2024-09-17T22:57:21.850258+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:21.850258+0000","last_undegraded":"2024-09-17T22:57:21.850258+0000","last_fullsized":"2024-09-17T22:57:21.850258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843441+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:20.843441+0000","last_peered":"2024-09-17T22:57:20.843441+0000","last_clean":"2024-09-17T22:57:20.843441+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:20.843441+0000","last_undegraded":"2024-09-17T22:57:20.843441+0000","last_fullsized":"2024-09-17T22:57:20.843441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845615+0000","last_change":"2024-09-17T22:57:21.845615+0000","last_active":"2024-09-17T22:57:21.845615+0000","last_peered":"2024-09-17T22:57:21.845615+0000","last_clean":"2024-09-17T22:57:21.845615+0000","last_became_active":"2024-09-17T22:57:19.848255+0000","last_became_peered":"2024-09-17T22:57:19.848255+0000","last_unstale":"2024-09-17T22:57:21.845615+0000","last_undegraded":"2024-09-17T22:57:21.845615+0000","last_fullsized":"2024-09-17T22:57:21.845615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:36.793896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083459400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845736+0000","last_change":"2024-09-17T22:57:21.845736+0000","last_active":"2024-09-17T22:57:21.845736+0000","last_peered":"2024-09-17T22:57:21.845736+0000","last_clean":"2024-09-17T22:57:21.845736+0000","last_became_active":"2024-09-17T22:57:19.951433+0000","last_became_peered":"2024-09-17T22:57:19.951433+0000","last_unstale":"2024-09-17T22:57:21.845736+0000","last_undegraded":"2024-09-17T22:57:21.845736+0000","last_fullsized":"2024-09-17T22:57:21.845736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:49.754345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045397599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:22.374427+0000","last_change":"2024-09-17T22:57:22.374427+0000","last_active":"2024-09-17T22:57:22.374427+0000","last_peered":"2024-09-17T22:57:22.374427+0000","last_clean":"2024-09-17T22:57:22.374427+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:57:22.374427+0000","last_undegraded":"2024-09-17T22:57:22.374427+0000","last_fullsized":"2024-09-17T22:57:22.374427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:34.792294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037622699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.845067+0000","last_change":"2024-09-17T22:57:21.845067+0000","last_active":"2024-09-17T22:57:21.845067+0000","last_peered":"2024-09-17T22:57:21.845067+0000","last_clean":"2024-09-17T22:57:21.845067+0000","last_became_active":"2024-09-17T22:57:19.848467+0000","last_became_peered":"2024-09-17T22:57:19.848467+0000","last_unstale":"2024-09-17T22:57:21.845067+0000","last_undegraded":"2024-09-17T22:57:21.845067+0000","last_fullsized":"2024-09-17T22:57:21.845067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:10.235230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056076400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.851244+0000","last_change":"2024-09-17T22:57:21.851244+0000","last_active":"2024-09-17T22:57:21.851244+0000","last_peered":"2024-09-17T22:57:21.851244+0000","last_clean":"2024-09-17T22:57:21.851244+0000","last_became_active":"2024-09-17T22:57:19.848596+0000","last_became_peered":"2024-09-17T22:57:19.848596+0000","last_unstale":"2024-09-17T22:57:21.851244+0000","last_undegraded":"2024-09-17T22:57:21.851244+0000","last_fullsized":"2024-09-17T22:57:21.851244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:52:23.592742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087459500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843537+0000","last_change":"2024-09-17T22:57:19.850040+0000","last_active":"2024-09-17T22:57:20.843537+0000","last_peered":"2024-09-17T22:57:20.843537+0000","last_clean":"2024-09-17T22:57:20.843537+0000","last_became_active":"2024-09-17T22:57:19.849680+0000","last_became_peered":"2024-09-17T22:57:19.849680+0000","last_unstale":"2024-09-17T22:57:20.843537+0000","last_undegraded":"2024-09-17T22:57:20.843537+0000","last_fullsized":"2024-09-17T22:57:20.843537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:27.646017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843657+0000","last_change":"2024-09-17T22:57:19.851935+0000","last_active":"2024-09-17T22:57:20.843657+0000","last_peered":"2024-09-17T22:57:20.843657+0000","last_clean":"2024-09-17T22:57:20.843657+0000","last_became_active":"2024-09-17T22:57:19.851650+0000","last_became_peered":"2024-09-17T22:57:19.851650+0000","last_unstale":"2024-09-17T22:57:20.843657+0000","last_undegraded":"2024-09-17T22:57:20.843657+0000","last_fullsized":"2024-09-17T22:57:20.843657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:09:26.822953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843665+0000","last_change":"2024-09-17T22:57:19.852024+0000","last_active":"2024-09-17T22:57:20.843665+0000","last_peered":"2024-09-17T22:57:20.843665+0000","last_clean":"2024-09-17T22:57:20.843665+0000","last_became_active":"2024-09-17T22:57:19.851755+0000","last_became_peered":"2024-09-17T22:57:19.851755+0000","last_unstale":"2024-09-17T22:57:20.843665+0000","last_undegraded":"2024-09-17T22:57:20.843665+0000","last_fullsized":"2024-09-17T22:57:20.843665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:19:34.321033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:22.374428+0000","last_change":"2024-09-17T22:57:22.374428+0000","last_active":"2024-09-17T22:57:22.374428+0000","last_peered":"2024-09-17T22:57:22.374428+0000","last_clean":"2024-09-17T22:57:22.374428+0000","last_became_active":"2024-09-17T22:57:19.853848+0000","last_became_peered":"2024-09-17T22:57:19.853848+0000","last_unstale":"2024-09-17T22:57:22.374428+0000","last_undegraded":"2024-09-17T22:57:22.374428+0000","last_fullsized":"2024-09-17T22:57:22.374428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:09.017221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036935199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843473+0000","last_change":"2024-09-17T22:57:19.851769+0000","last_active":"2024-09-17T22:57:20.843473+0000","last_peered":"2024-09-17T22:57:20.843473+0000","last_clean":"2024-09-17T22:57:20.843473+0000","last_became_active":"2024-09-17T22:57:19.851389+0000","last_became_peered":"2024-09-17T22:57:19.851389+0000","last_unstale":"2024-09-17T22:57:20.843473+0000","last_undegraded":"2024-09-17T22:57:20.843473+0000","last_fullsized":"2024-09-17T22:57:20.843473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:26.017698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:57:21.855983+0000","last_change":"2024-09-17T22:57:21.855983+0000","last_active":"2024-09-17T22:57:21.855983+0000","last_peered":"2024-09-17T22:57:21.855983+0000","last_clean":"2024-09-17T22:57:21.855983+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:57:21.855983+0000","last_undegraded":"2024-09-17T22:57:21.855983+0000","last_fullsized":"2024-09-17T22:57:21.855983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:15.133118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001538125,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T22:57:20.843539+0000","last_change":"2024-09-17T22:57:19.852627+0000","last_active":"2024-09-17T22:57:20.843539+0000","last_peered":"2024-09-17T22:57:20.843539+0000","last_clean":"2024-09-17T22:57:20.843539+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:57:20.843539+0000","last_undegraded":"2024-09-17T22:57:20.843539+0000","last_fullsized":"2024-09-17T22:57:20.843539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:50.185638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":89839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":96069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":96053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":89839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":89839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":253952,"data_stored":89866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":686448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":674096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:24.896 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [5, 2] 2024-09-17T22:57:24.896 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.11', '5', '2'] 2024-09-17T22:57:24.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.11 5 2 2024-09-17T22:57:24.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:24.938 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:24.964 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:57:24.964 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:24.962+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:25.031 INFO:teuthology.orchestra.run.smithi159.stdout:flattening image image_client.0-clone1 2024-09-17T22:57:25.061 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:25.058+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:25.085 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:25.085 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:25.089 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:25.089 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:25.097 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:25.122 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:25.123 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:25.156 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:25.156 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:25.161 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:25.159+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:25.187 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:25.187 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:25.204 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T22:57:25.206 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:25.206 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:25.223 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:25.242 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:25.243 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:25.253 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:25.253 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:25.253 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:25.253 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:25.263 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:25.263 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:25.260+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:25.264 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:25.265 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:25.345 INFO:teuthology.orchestra.run.smithi159.stdout:126 writesame 0x32a9ce6 thru 0x32b2eab data_size 0x2(0x91c6 bytes) 2024-09-17T22:57:25.360 INFO:teuthology.orchestra.run.smithi159.stdout:127 punch from 0xbc7b27c to 0xbc7c52d, (0x12b1 bytes) 2024-09-17T22:57:25.360 INFO:teuthology.orchestra.run.smithi159.stdout:129 write 0x429d1e9 thru 0x42a1736 (0x454e bytes) 2024-09-17T22:57:25.363 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:25.361+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:25.364 INFO:teuthology.orchestra.run.smithi159.stdout:130 writesame 0x5f74e5e thru 0x5f75867 data_size 0x5(0xa0a bytes) 2024-09-17T22:57:25.367 INFO:teuthology.orchestra.run.smithi159.stdout:131 write 0x34a8584 thru 0x34afea8 (0x7925 bytes) 2024-09-17T22:57:25.370 INFO:teuthology.orchestra.run.smithi159.stdout:133 read 0x82be47c thru 0x82ce2e1 (0xfe66 bytes) 2024-09-17T22:57:25.371 INFO:teuthology.orchestra.run.smithi159.stdout:134 writesame 0x96e92f thru 0x97a6f3 data_size 0xd(0xbdc5 bytes) 2024-09-17T22:57:25.374 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:25.374 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:25.374 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:25.374 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:25.374 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:25.382 INFO:teuthology.orchestra.run.smithi159.stdout:135 trunc from 0xe990eb1 to 0x913ff1b 2024-09-17T22:57:25.387 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:25.393 INFO:teuthology.orchestra.run.smithi159.stdout:136 read 0x35c8d0f thru 0x35cdbf7 (0x4ee9 bytes) 2024-09-17T22:57:25.393 INFO:teuthology.orchestra.run.smithi159.stdout:137 punch from 0x7b52440 to 0x7b5da51, (0xb611 bytes) 2024-09-17T22:57:25.393 INFO:teuthology.orchestra.run.smithi159.stdout:138 punch from 0x1ee2944 to 0x1ee84a3, (0x5b5f bytes) 2024-09-17T22:57:25.393 INFO:teuthology.orchestra.run.smithi159.stdout:141 writesame 0x3bb84e1 thru 0x3bbedc2 data_size 0x1e(0x68e2 bytes) 2024-09-17T22:57:25.404 INFO:teuthology.orchestra.run.smithi159.stdout:142 writesame 0xb1f1790 thru 0xb1f19d5 data_size 0x3(0x246 bytes) 2024-09-17T22:57:25.404 INFO:teuthology.orchestra.run.smithi159.stdout:143 writesame 0x6b612e8 thru 0x6b6b253 data_size 0x4c(0x9f6c bytes) 2024-09-17T22:57:25.407 INFO:teuthology.orchestra.run.smithi159.stdout:144 punch from 0x1117321 to 0x111afb7, (0x3c96 bytes) 2024-09-17T22:57:25.417 INFO:teuthology.orchestra.run.smithi159.stdout:145 write 0xe74c11c thru 0xe74e8df (0x27c4 bytes) 2024-09-17T22:57:25.432 INFO:teuthology.orchestra.run.smithi159.stdout:146 trunc from 0xe74e8e0 to 0x48d01eb 2024-09-17T22:57:25.448 INFO:teuthology.orchestra.run.smithi159.stdout:147 read 0x26d1d5d thru 0x26d3114 (0x13b8 bytes) 2024-09-17T22:57:25.454 INFO:teuthology.orchestra.run.smithi159.stdout:148 trunc from 0x48d01eb to 0x945730f 2024-09-17T22:57:25.459 INFO:teuthology.orchestra.run.smithi159.stdout:149 punch from 0x5c10d04 to 0x5c18a4b, (0x7d47 bytes) 2024-09-17T22:57:25.460 INFO:teuthology.orchestra.run.smithi159.stdout:150 read 0x57845d2 thru 0x5788111 (0x3b40 bytes) 2024-09-17T22:57:25.460 INFO:teuthology.orchestra.run.smithi159.stdout:151 punch from 0x37d37f9 to 0x37dc7c9, (0x8fd0 bytes) 2024-09-17T22:57:25.465 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:25.463+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:25.469 INFO:teuthology.orchestra.run.smithi159.stdout:153 write 0xd639148 thru 0xd64144f (0x8308 bytes) 2024-09-17T22:57:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:154 read 0x5cf2548 thru 0x5cf4bf9 (0x26b2 bytes) 2024-09-17T22:57:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:155 write 0x75ad6a4 thru 0x75bc5cd (0xef2a bytes) 2024-09-17T22:57:25.476 INFO:teuthology.orchestra.run.smithi159.stdout:156 write 0xabf8bb1 thru 0xabfdbc4 (0x5014 bytes) 2024-09-17T22:57:25.484 INFO:teuthology.orchestra.run.smithi159.stdout:157 trunc from 0xd641450 to 0x956ece7 2024-09-17T22:57:25.495 INFO:teuthology.orchestra.run.smithi159.stdout:158 write 0x45aa3c6 thru 0x45b286c (0x84a7 bytes) 2024-09-17T22:57:25.498 INFO:teuthology.orchestra.run.smithi159.stdout:159 read 0x250ce0b thru 0x2514817 (0x7a0d bytes) 2024-09-17T22:57:25.499 INFO:teuthology.orchestra.run.smithi159.stdout:161 write 0x2e1bcc8 thru 0x2e27b4f (0xbe88 bytes) 2024-09-17T22:57:25.503 INFO:teuthology.orchestra.run.smithi159.stdout:163 punch from 0x8a3aec3 to 0x8a40eb3, (0x5ff0 bytes) 2024-09-17T22:57:25.506 INFO:teuthology.orchestra.run.smithi159.stdout:166 trunc from 0x956ece7 to 0x706360c 2024-09-17T22:57:25.516 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:25.516 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.619769674, 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00080673799999999999, 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893409+0000", 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893419+0000", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8020000000000003e-06 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893428+0000", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2080000000000006e-06 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893437+0000", 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3649999999999993e-06 2024-09-17T22:57:25.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893496+0000", 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8489999999999997e-05 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.893523+0000", 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7055000000000001e-05 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.894177+0000", 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00065433799999999997 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:10.894215+0000", 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.82e-05 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:25.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.613068499000001, 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098273299999999996, 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:25.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900110+0000", 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.36e-07 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900118+0000", 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0099999999999995e-06 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900155+0000", 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6925000000000003e-05 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900169+0000", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3723000000000001e-05 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900230+0000", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1554000000000001e-05 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.900249+0000", 2024-09-17T22:57:25.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8303000000000001e-05 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.901037+0000", 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00078825800000000001 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:11.901093+0000", 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5723999999999997e-05 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7 3.1a e24/22 3:5cea7035:::rbd_info:head v 24'1, mlcod=0'0)", 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.667116321, 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000673235, 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846062+0000", 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9000000000000001e-07 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846067+0000", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8910000000000003e-06 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846077+0000", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8120000000000002e-06 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846084+0000", 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3329999999999999e-06 2024-09-17T22:57:25.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846127+0000", 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2845000000000001e-05 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846149+0000", 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2016999999999999e-05 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846697+0000", 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054781100000000004 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:20.846735+0000", 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8336e-05 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:9 3.1a e25/22 3:5cea7035:::rbd_info:head v 25'2, mlcod=0'0)", 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:25.531 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.6670124360000003, 2024-09-17T22:57:25.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017871269999999999, 2024-09-17T22:57:25.532 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.532 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.532 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:25.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846166+0000", 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846170+0000", 2024-09-17T22:57:25.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6770000000000001e-06 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846179+0000", 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9619999999999999e-06 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846188+0000", 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1379999999999996e-06 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846241+0000", 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3625000000000002e-05 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.846258+0000", 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6730000000000001e-05 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.847904+0000", 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016458569999999999 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:21.847953+0000", 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8908e-05 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:657 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.173389+0000", 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33978918000000002, 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001266999, 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.539 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 657 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173389+0000", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173389+0000", 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173392+0000", 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173406+0000", 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3929e-05 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173432+0000", 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6319999999999999e-05 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173447+0000", 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4262e-05 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173519+0000", 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2166e-05 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173589+0000", 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0588999999999998e-05 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 2", 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173712+0000", 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000122879 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174331+0000", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061899500000000001 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174588+0000", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025712100000000002 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174623+0000", 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4329000000000003e-05 2024-09-17T22:57:25.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174656+0000", 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3497999999999997e-05 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:658 3.0 3:0066e439:::rbd_header.1188f4a20928:head [notify cookie 139917888250304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.174745+0000", 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33843348699999998, 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000182029, 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.546 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 658 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174745+0000", 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174745+0000", 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174746+0000", 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0100000000000001e-06 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174753+0000", 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0859999999999996e-06 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174765+0000", 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1919000000000001e-05 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174772+0000", 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.939e-06 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174806+0000", 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4439e-05 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174861+0000", 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5059999999999998e-05 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174927+0000", 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5575999999999997e-05 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:659 3.0 3:0066e439:::rbd_header.1188f4a20928:head [watch unwatch cookie 139918089218928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.174796+0000", 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33838253600000001, 2024-09-17T22:57:25.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0024400260000000001, 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.552 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 659 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174796+0000", 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174796+0000", 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174796+0000", 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1500000000000004e-07 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174800+0000", 2024-09-17T22:57:25.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0539999999999996e-06 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174809+0000", 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5809999999999997e-06 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174815+0000", 2024-09-17T22:57:25.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7690000000000002e-06 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174953+0000", 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013816100000000001 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175010+0000", 2024-09-17T22:57:25.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6814000000000002e-05 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 2", 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175104+0000", 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3820000000000006e-05 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.176963+0000", 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018592319999999999 2024-09-17T22:57:25.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177120+0000", 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000156844 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177153+0000", 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3160000000000001e-05 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177236+0000", 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2976e-05 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:660 3.0 3:0066e439:::rbd_header.1188f4a20928:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.174989+0000", 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33818922699999998, 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023653139999999999, 2024-09-17T22:57:25.559 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 660 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174989+0000", 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174989+0000", 2024-09-17T22:57:25.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174990+0000", 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.92e-07 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174994+0000", 2024-09-17T22:57:25.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5469999999999998e-06 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175004+0000", 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2410000000000001e-06 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175021+0000", 2024-09-17T22:57:25.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7215999999999999e-05 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175302+0000", 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00028160400000000002 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for rw locks", 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175387+0000", 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4500999999999996e-05 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177267+0000", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001880009 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177308+0000", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0890999999999999e-05 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177354+0000", 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6513e-05 2024-09-17T22:57:25.565 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop_reply(client.4488.0:659 3.0 e27/22 ondisk, result = 0)", 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.177043+0000", 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.336135456, 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019794800000000001, 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177043+0000", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177043+0000", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177044+0000", 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3200000000000004e-07 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177048+0000", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3610000000000003e-06 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177061+0000", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2826000000000001e-05 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177068+0000", 2024-09-17T22:57:25.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0199999999999997e-06 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177106+0000", 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7994999999999998e-05 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177114+0000", 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7180000000000002e-06 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177241+0000", 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000126296 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:661 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.179753+0000", 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33342500200000003, 2024-09-17T22:57:25.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00018018, 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 661 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179753+0000", 2024-09-17T22:57:25.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179753+0000", 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179754+0000", 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0580000000000001e-06 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179759+0000", 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.95e-06 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179770+0000", 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1002000000000001e-05 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179785+0000", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4273e-05 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179834+0000", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9486e-05 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179881+0000", 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.676e-05 2024-09-17T22:57:25.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.179933+0000", 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2651000000000001e-05 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:662 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.180025+0000", 2024-09-17T22:57:25.575 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.33315320500000001, 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019229199999999999, 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 662 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180025+0000", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180025+0000", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180026+0000", 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180030+0000", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1289999999999998e-06 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180038+0000", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8010000000000004e-06 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180042+0000", 2024-09-17T22:57:25.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4789999999999998e-06 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180075+0000", 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3201999999999998e-05 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180121+0000", 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5627999999999997e-05 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.180217+0000", 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6427000000000004e-05 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:674 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.188130+0000", 2024-09-17T22:57:25.580 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.325047945, 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00031692200000000003, 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 674 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188130+0000", 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188130+0000", 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188133+0000", 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:57:25.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188144+0000", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1219999999999999e-05 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188165+0000", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1239000000000001e-05 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188176+0000", 2024-09-17T22:57:25.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0804e-05 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188235+0000", 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8461000000000002e-05 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188298+0000", 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3749000000000002e-05 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188447+0000", 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014886500000000001 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:675 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.188541+0000", 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32463717399999997, 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019520599999999999, 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.585 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 675 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188541+0000", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188541+0000", 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188542+0000", 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9699999999999995e-07 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188546+0000", 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188558+0000", 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1139e-05 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188564+0000", 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7970000000000003e-06 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188603+0000", 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8359000000000003e-05 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188641+0000", 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8720000000000002e-05 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.188736+0000", 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4835999999999998e-05 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.589 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:677 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.189060+0000", 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32411837799999998, 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00017785899999999999, 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 677 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189060+0000", 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189060+0000", 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189061+0000", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189065+0000", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1289999999999998e-06 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189075+0000", 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0297e-05 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189081+0000", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189115+0000", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4493999999999999e-05 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189156+0000", 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0361999999999997e-05 2024-09-17T22:57:25.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189238+0000", 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1957000000000002e-05 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:678 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.189314+0000", 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32386434200000003, 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00031636000000000001, 2024-09-17T22:57:25.594 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 678 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189314+0000", 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189314+0000", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189315+0000", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4199999999999995e-07 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189319+0000", 2024-09-17T22:57:25.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-06 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189328+0000", 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8709999999999992e-06 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189348+0000", 2024-09-17T22:57:25.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0026000000000001e-05 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189404+0000", 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5554999999999999e-05 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189493+0000", 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9184999999999994e-05 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189630+0000", 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013715099999999999 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:679 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.189945+0000", 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.323233153, 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00031687399999999998, 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.599 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 679 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189945+0000", 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189945+0000", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189946+0000", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4900000000000005e-07 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189950+0000", 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4630000000000003e-06 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189961+0000", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.058e-05 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.189967+0000", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3770000000000001e-06 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190003+0000", 2024-09-17T22:57:25.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5302000000000002e-05 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190060+0000", 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7108999999999998e-05 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190262+0000", 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000202294 2024-09-17T22:57:25.603 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:680 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.190351+0000", 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32282723499999999, 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025534400000000001, 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 680 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190351+0000", 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190351+0000", 2024-09-17T22:57:25.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190352+0000", 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5899999999999995e-07 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190356+0000", 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.216e-06 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190367+0000", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0886e-05 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190373+0000", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2829999999999998e-06 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190409+0000", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5750999999999997e-05 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190454+0000", 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4923999999999999e-05 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.190606+0000", 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00015252500000000001 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.608 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:683 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.191369+0000", 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32180927199999998, 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00017400599999999999, 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 683 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191369+0000", 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191369+0000", 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191370+0000", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6400000000000001e-07 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191374+0000", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2549999999999999e-06 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191385+0000", 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1168999999999999e-05 2024-09-17T22:57:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191391+0000", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-06 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191432+0000", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0672e-05 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191476+0000", 2024-09-17T22:57:25.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3399999999999998e-05 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191543+0000", 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7451000000000002e-05 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:684 3.0 3:0066e439:::rbd_header.1188f4a20928:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.191628+0000", 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.32154994999999997, 2024-09-17T22:57:25.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000263923, 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 684 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191628+0000", 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191628+0000", 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191629+0000", 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.61e-07 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191633+0000", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0679999999999998e-06 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191643+0000", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0329000000000001e-05 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191650+0000", 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0900000000000001e-06 2024-09-17T22:57:25.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191682+0000", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2888000000000001e-05 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191727+0000", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4091000000000002e-05 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.191892+0000", 2024-09-17T22:57:25.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00016569600000000001 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:725 3.4 3:248b9aac:::rbd_data.1188676ee382.0000000000000019:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.232948+0000", 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.28023022800000003, 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00022608900000000001, 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:25.618 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 725 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232948+0000", 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:25.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232948+0000", 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232950+0000", 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0339999999999999e-06 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232959+0000", 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6559999999999998e-06 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232986+0000", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6784e-05 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.232997+0000", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1262e-05 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.233072+0000", 2024-09-17T22:57:25.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5068999999999996e-05 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.233174+0000", 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102284 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:25.623 INFO:teuthology.orchestra.run.smithi159.stdout:168 trunc from 0x706360c to 0x385fb1b 2024-09-17T22:57:25.624 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:25.625 INFO:teuthology.orchestra.run.smithi159.stdout:169 read 0x21909d6 thru 0x219fbab (0xf1d6 bytes) 2024-09-17T22:57:25.625 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:25.625 INFO:teuthology.orchestra.run.smithi159.stdout:172 clone 3 order 23 su 4194304 sc 3 2024-09-17T22:57:25.625 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:25.622+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:25.724 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:25.722+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:25.768 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:25.769 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:25.769 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:25.769 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:25.777 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:25.825 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:25.823+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:25.880 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:25.881 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:25.881 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:25.881 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:25.881 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:25.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:25.926 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:25.924+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:25.959 INFO:teuthology.orchestra.run.smithi159.stderr:set 3.11 pg_upmap mapping to [5,2] 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:26.003 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 16.109914219, 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:26.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:657 3.0 e27/22 3:0066e439:::rbd_header.1188f4a20928:head v 27'25, mlcod=27'23)", 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.173798+0000", 2024-09-17T22:57:26.009 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.82709584800000002, 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00066787699999999999, 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173798+0000", 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173798+0000", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173800+0000", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3689999999999999e-06 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173812+0000", 2024-09-17T22:57:26.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2889e-05 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173832+0000", 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9678999999999999e-05 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173847+0000", 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4877e-05 2024-09-17T22:57:26.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173922+0000", 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5242999999999998e-05 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.173948+0000", 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5363000000000001e-05 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174429+0000", 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00048121999999999998 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.174466+0000", 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7237000000000003e-05 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.014 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:659 3.0 e27/22 3:0066e439:::rbd_header.1188f4a20928:head v 27'26, mlcod=27'24)", 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.175165+0000", 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.82572920400000005, 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001838688, 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175165+0000", 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175165+0000", 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175166+0000", 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4300000000000002e-07 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175172+0000", 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6290000000000004e-06 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175184+0000", 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1719000000000001e-05 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175191+0000", 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0029999999999997e-06 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175239+0000", 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.816e-05 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.175259+0000", 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0166999999999999e-05 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.176958+0000", 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001699089 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.177004+0000", 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5178e-05 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:783 3.4 e27/22 3:24848ee1:::rbd_data.118840f89003.00000000000000ef:head v 27'3, mlcod=25'1)", 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.394651+0000", 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.60624355200000002, 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00086269799999999996, 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.021 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394651+0000", 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394651+0000", 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394652+0000", 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394679+0000", 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7702999999999999e-05 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394695+0000", 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5625999999999999e-05 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394702+0000", 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:57:26.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394754+0000", 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2222000000000002e-05 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.394770+0000", 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909e-05 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.395480+0000", 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00070970799999999998 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.395513+0000", 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3615999999999998e-05 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.026 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:899 3.4 e27/22 3:24848ee1:::rbd_data.118840f89003.00000000000000ef:head v 27'4, mlcod=25'2)", 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:25.520612+0000", 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.48028177900000002, 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00069036300000000004, 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520612+0000", 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520612+0000", 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520614+0000", 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.24e-06 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520622+0000", 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.439e-06 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520641+0000", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8643000000000001e-05 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520650+0000", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2690000000000005e-06 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520738+0000", 2024-09-17T22:57:26.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8506999999999998e-05 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.520760+0000", 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1231000000000001e-05 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.521255+0000", 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00049573100000000001 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:25.521303+0000", 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7302999999999998e-05 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.033 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.033 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:26.035 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:26.035 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone2 from 0x385fb1b (overlap 0x385fb1b) to 0x21d3043 2024-09-17T22:57:26.036 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:26.031+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:26.135 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:26.133+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:26.185 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:26.185 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:26.185 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:26.185 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:26.194 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:26.237 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:26.234+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:26.337 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:26.335+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:26.337 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:26.337 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:26.338 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:26.338 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:26.338 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:26.346 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:26.438 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:26.435+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:26.450 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:26.450 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:26.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1029 3.19 e28/22 3:9e020cf5:::rbd_data.118840f89003.000000000000008e:head v 28'4, mlcod=27'1)", 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.42054850199999999, 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051679889999999996, 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026576+0000", 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026582+0000", 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6200000000000001e-06 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026592+0000", 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9040000000000004e-06 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026599+0000", 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026643+0000", 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4719999999999999e-05 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026664+0000", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1058000000000001e-05 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031712+0000", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0050476200000000001 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031743+0000", 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1031000000000002e-05 2024-09-17T22:57:26.455 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1031 3.15 e28/22 3:aa926c06:::rbd_data.118840f89003.000000000000008c:head v 28'6, mlcod=26'4)", 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.42020281199999998, 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048447150000000003, 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.456 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3300000000000001e-07 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026927+0000", 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2120000000000002e-06 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026934+0000", 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.255e-06 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026939+0000", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9740000000000001e-06 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026979+0000", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0130999999999999e-05 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026991+0000", 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:57:26.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031727+0000", 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047360659999999997 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031765+0000", 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8515e-05 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1107 3.8 3:1296b2d3:::rbd_data.118840f89003.00000000000000a4:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.40830182700000001, 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005694859, 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1107 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.461 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038826+0000", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3349999999999998e-06 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038833+0000", 2024-09-17T22:57:26.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4279999999999999e-06 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038839+0000", 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8080000000000001e-06 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039055+0000", 2024-09-17T22:57:26.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000216157 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039144+0000", 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9210999999999997e-05 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044517+0000", 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053722930000000002 2024-09-17T22:57:26.465 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 150 entries 28'8 (0'0) error 3:a8b1de66:::rbd_data.118840f89003.00000000000000a6:head by client.4488.0:1112 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.40739065099999999, 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047563509999999998, 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.466 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039734+0000", 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039743+0000", 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5230000000000001e-06 2024-09-17T22:57:26.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039762+0000", 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9324999999999999e-05 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039775+0000", 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2564e-05 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039833+0000", 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7954000000000001e-05 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044489+0000", 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046562790000000001 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1122 3.15 e28/22 3:a9e9a37d:::rbd_data.118840f89003.00000000000000a0:head v 28'9, mlcod=28'6)", 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.40432649599999998, 2024-09-17T22:57:26.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00465505, 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042798+0000", 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.82e-07 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042802+0000", 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3030000000000001e-06 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042814+0000", 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2289999999999999e-05 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042821+0000", 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4049999999999997e-06 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042868+0000", 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7605999999999998e-05 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042882+0000", 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3464e-05 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047413+0000", 2024-09-17T22:57:26.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045310159999999997 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047452+0000", 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9384000000000001e-05 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1132 3.8 3:1296b2d3:::rbd_data.118840f89003.00000000000000a4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.40214790299999997, 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053778960000000001, 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1132 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1900000000000001e-07 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044981+0000", 2024-09-17T22:57:26.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044992+0000", 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0679000000000001e-05 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044999+0000", 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045085+0000", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6337999999999999e-05 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045142+0000", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7055000000000002e-05 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045268+0000", 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00012598899999999999 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050158+0000", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048892739999999999 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050285+0000", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00012700799999999999 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050331+0000", 2024-09-17T22:57:26.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6696999999999997e-05 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050354+0000", 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2096e-05 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1133 3.15 e28/22 3:a8b1de66:::rbd_data.118840f89003.00000000000000a6:head v 28'10, mlcod=28'7)", 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:26.484 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.40154862800000002, 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045667720000000002, 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045576+0000", 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045581+0000", 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-06 2024-09-17T22:57:26.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045590+0000", 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0990000000000005e-06 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045594+0000", 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045638+0000", 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3658000000000001e-05 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045651+0000", 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.253e-05 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050102+0000", 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004451048 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050142+0000", 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9993000000000001e-05 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1136 3.7 3:e2c60e86:::rbd_data.118840f89003.00000000000000a9:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.39974057699999999, 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0056381740000000001, 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.490 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1136 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047384+0000", 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8599999999999997e-07 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047490+0000", 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106618 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047505+0000", 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4175999999999999e-05 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047516+0000", 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1396999999999999e-05 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047584+0000", 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7830000000000006e-05 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047731+0000", 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014677000000000001 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.053021+0000", 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052904969999999999 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1145 3.b 3:d691d729:::rbd_data.118840f89003.00000000000000ad:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.39930246400000002, 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053637859999999997, 2024-09-17T22:57:26.495 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1145 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:26.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047822+0000", 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6700000000000003e-07 2024-09-17T22:57:26.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047825+0000", 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0110000000000001e-06 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047833+0000", 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5040000000000002e-06 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047839+0000", 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7049999999999996e-06 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047892+0000", 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3168999999999999e-05 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048034+0000", 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014156499999999999 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.053185+0000", 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051512650000000004 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 153 entries 28'11 (0'0) error 3:af9fb172:::rbd_data.118840f89003.00000000000000ab:head by client.4488.0:1140 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'8)", 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.39885034200000002, 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046930879999999998, 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048274+0000", 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:26.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048281+0000", 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3529999999999997e-06 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048300+0000", 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8899e-05 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048313+0000", 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3254999999999999e-05 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048373+0000", 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9873000000000001e-05 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.052966+0000", 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045932689999999997 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1148 3.8 3:11bb6454:::rbd_data.118840f89003.00000000000000ae:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.396862669, 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005917276, 2024-09-17T22:57:26.505 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1148 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050262+0000", 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6799999999999999e-07 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050265+0000", 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1030000000000002e-06 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050274+0000", 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5130000000000002e-06 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050281+0000", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6239999999999996e-06 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050397+0000", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000116493 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050483+0000", 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5624000000000006e-05 2024-09-17T22:57:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056178+0000", 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0056951509999999999 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1179 3.b 3:d7b125d7:::rbd_data.118840f89003.00000000000000b6:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.39051068, 2024-09-17T22:57:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051194329999999996, 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1179 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056614+0000", 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0399999999999995e-07 2024-09-17T22:57:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056617+0000", 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6440000000000002e-06 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056626+0000", 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3440000000000001e-06 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056632+0000", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056706+0000", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4538999999999999e-05 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056844+0000", 2024-09-17T22:57:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013722699999999999 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061732+0000", 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048886759999999998 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1180 3.8 3:11bb6454:::rbd_data.118840f89003.00000000000000ae:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.390286154, 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049744419999999999, 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1180 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056838+0000", 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5300000000000002e-07 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056843+0000", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9350000000000002e-06 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056856+0000", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.295e-05 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056873+0000", 2024-09-17T22:57:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6410999999999999e-05 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056932+0000", 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9506000000000003e-05 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.057001+0000", 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9144000000000001e-05 2024-09-17T22:57:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.057191+0000", 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00018997299999999999 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061466+0000", 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0042747660000000002 2024-09-17T22:57:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061722+0000", 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025618700000000001 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061774+0000", 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2021999999999998e-05 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061812+0000", 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7595000000000002e-05 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1207 3.19 e28/22 3:9df5c97b:::rbd_data.118840f89003.00000000000000b7:head v 28'9, mlcod=28'6)", 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.38245500700000001, 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048842470000000004, 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4600000000000004e-07 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064675+0000", 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6690000000000003e-06 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064686+0000", 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1518999999999999e-05 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064691+0000", 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6149999999999999e-06 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064734+0000", 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3343000000000002e-05 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064750+0000", 2024-09-17T22:57:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6013999999999999e-05 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069520+0000", 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047699270000000002 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069553+0000", 2024-09-17T22:57:26.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2413999999999998e-05 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1222 3.b 3:d3891ddd:::rbd_data.118840f89003.00000000000000c2:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.37729826999999999, 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005307137, 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.528 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1222 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069826+0000", 2024-09-17T22:57:26.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8599999999999998e-07 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069830+0000", 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7730000000000001e-06 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069840+0000", 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0318e-05 2024-09-17T22:57:26.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069845+0000", 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069882+0000", 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7070999999999998e-05 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069956+0000", 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3693000000000001e-05 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075132+0000", 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051764039999999999 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.533 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1240 3.b 3:d3891ddd:::rbd_data.118840f89003.00000000000000c2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.37155032900000001, 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048908099999999998, 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:26.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1240 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075574+0000", 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075579+0000", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.955e-06 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075588+0000", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075594+0000", 2024-09-17T22:57:26.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9549999999999997e-06 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075637+0000", 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3377999999999997e-05 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075688+0000", 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810999999999999e-05 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075804+0000", 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011627499999999999 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080141+0000", 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0043364249999999997 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080358+0000", 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021778800000000001 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080422+0000", 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899000000000006e-05 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080464+0000", 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1625000000000002e-05 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.c epoch 28/22 rep_tid 36 entries 28'6 (0'0) error 3:30186e42:::rbd_data.118840f89003.00000000000000cd:head by client.4488.0:1259 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'3)", 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.36355151899999999, 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047667910000000003, 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083573+0000", 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9699999999999996e-07 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083580+0000", 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0960000000000003e-06 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083595+0000", 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5591999999999999e-05 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083604+0000", 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1519999999999998e-06 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083672+0000", 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7488000000000002e-05 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.088339+0000", 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046668660000000004 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.544 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.19 epoch 28/22 rep_tid 45 entries 28'12 (0'0) error 3:9f6cdf93:::rbd_data.118840f89003.00000000000000ce:head by client.4488.0:1266 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'10)", 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.36090381199999999, 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0050898829999999999, 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086221+0000", 2024-09-17T22:57:26.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-07 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086229+0000", 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0949999999999996e-06 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086248+0000", 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9440999999999999e-05 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086262+0000", 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3709999999999999e-05 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086342+0000", 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9957999999999996e-05 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.091310+0000", 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049677719999999996 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1287 3.19 e28/22 3:9f6cdf93:::rbd_data.118840f89003.00000000000000ce:head v 28'13, mlcod=28'11)", 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.35301467600000003, 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0055062899999999996, 2024-09-17T22:57:26.549 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094110+0000", 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4799999999999997e-07 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094115+0000", 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1839999999999998e-06 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094127+0000", 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1839e-05 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094141+0000", 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4078e-05 2024-09-17T22:57:26.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094190+0000", 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8984999999999999e-05 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:26.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094208+0000", 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8583000000000001e-05 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099526+0000", 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053173450000000002 2024-09-17T22:57:26.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099615+0000", 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9528000000000001e-05 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.557 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 167 entries 28'15 (0'0) error 3:ac37f788:::rbd_data.118840f89003.00000000000000d5:head by client.4488.0:1293 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'13)", 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.35215628999999998, 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046150039999999998, 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:26.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094968+0000", 2024-09-17T22:57:26.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.16e-07 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094976+0000", 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8639999999999996e-06 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094996+0000", 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0458999999999998e-05 2024-09-17T22:57:26.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.095021+0000", 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4258000000000001e-05 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.095084+0000", 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3025000000000001e-05 2024-09-17T22:57:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099582+0000", 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0044985820000000001 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:26.562 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:26.564 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:26.565 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:26.562+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:26.664 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:26.662+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:26.670 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:26.670 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:26.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:26.765 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:26.763+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:26.821 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:26.822 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:26.822 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:26.822 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:26.822 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:26.830 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:26.874 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:26.872+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:26.935 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:26.935 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1044 3.9 3:954c0790:::rbd_data.118840f89003.0000000000000090:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.904121858, 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045157670000000004, 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1044 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027771+0000", 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.08e-06 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027776+0000", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5260000000000001e-06 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027791+0000", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4222e-05 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027798+0000", 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9109999999999993e-06 2024-09-17T22:57:26.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027854+0000", 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5788000000000002e-05 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027941+0000", 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7026000000000003e-05 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028149+0000", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020735299999999999 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.029649+0000", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015007219999999999 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032187+0000", 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025375430000000002 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032244+0000", 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6938000000000003e-05 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032285+0000", 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1658e-05 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1102 3.1d e28/22 3:b8b32f6b:::rbd_data.118840f89003.000000000000009e:head v 28'7, mlcod=28'5)", 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.89348770399999999, 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052004240000000004, 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3e-07 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038418+0000", 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3531e-05 2024-09-17T22:57:26.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038428+0000", 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7990000000000005e-06 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038435+0000", 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7610000000000002e-06 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038481+0000", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6449000000000002e-05 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038496+0000", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.485e-05 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043569+0000", 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050729190000000004 2024-09-17T22:57:26.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043604+0000", 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5485000000000003e-05 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.1 epoch 28/22 rep_tid 108 entries 28'3 (0'0) error 3:82735225:::rbd_data.118840f89003.00000000000000a3:head by client.4488.0:1104 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'1)", 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.89295765999999999, 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049701559999999999, 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.947 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038935+0000", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4000000000000001e-07 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038941+0000", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5679999999999996e-06 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038956+0000", 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4800000000000001e-05 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038965+0000", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8629999999999997e-06 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039026+0000", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1566e-05 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043904+0000", 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048776189999999997 2024-09-17T22:57:26.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1110 3.9 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.89278942299999997, 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082675690000000007, 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1110 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039103+0000", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8800000000000001e-07 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039109+0000", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.711e-06 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039120+0000", 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1082e-05 2024-09-17T22:57:26.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039128+0000", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7540000000000006e-06 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039168+0000", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0373000000000001e-05 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039262+0000", 2024-09-17T22:57:26.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3875999999999994e-05 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.047370+0000", 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081078850000000004 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1115 3.1a 3:581e07df:::rbd_data.118840f89003.000000000000009f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.89223992200000002, 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050351440000000001, 2024-09-17T22:57:26.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1115 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:26.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.47e-07 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039657+0000", 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7520000000000004e-06 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039669+0000", 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.134e-05 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039676+0000", 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7400000000000004e-06 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039735+0000", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9190000000000001e-05 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039818+0000", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2084999999999998e-05 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039937+0000", 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000119592 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044203+0000", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659890000000004 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044621+0000", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041764299999999999 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044653+0000", 2024-09-17T22:57:26.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2239000000000001e-05 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044687+0000", 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3726999999999997e-05 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1147 3.1a 3:596d8795:::rbd_data.118840f89003.00000000000000a7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.048640+0000", 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88325119500000004, 2024-09-17T22:57:26.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045142419999999999, 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1147 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048640+0000", 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048640+0000", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048641+0000", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048649+0000", 2024-09-17T22:57:26.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048660+0000", 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.098e-05 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048668+0000", 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4800000000000004e-06 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048729+0000", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1587999999999994e-05 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048801+0000", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1506000000000003e-05 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.048924+0000", 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012287100000000001 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052646+0000", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0037218849999999999 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053056+0000", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041035099999999999 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053115+0000", 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9512999999999998e-05 2024-09-17T22:57:26.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053155+0000", 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9144000000000003e-05 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1152 3.3 3:c6fe237e:::rbd_data.118840f89003.00000000000000b0:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88145466900000002, 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055420089999999997, 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1152 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050438+0000", 2024-09-17T22:57:26.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2500000000000004e-07 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050442+0000", 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3470000000000001e-06 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050454+0000", 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1847e-05 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050462+0000", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050517+0000", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5124000000000003e-05 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050646+0000", 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012879800000000001 2024-09-17T22:57:26.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.055979+0000", 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053330620000000004 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.a epoch 28/22 rep_tid 26 entries 28'15 (0'0) error 3:5642bcd0:::rbd_data.118840f89003.00000000000000af:head by client.4488.0:1151 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'13)", 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.88109621400000004, 2024-09-17T22:57:26.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045632449999999996, 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050796+0000", 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050803+0000", 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050820+0000", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6798000000000001e-05 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050834+0000", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4239e-05 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050907+0000", 2024-09-17T22:57:26.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3351999999999998e-05 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.055359+0000", 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044514699999999999 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.1c epoch 28/22 rep_tid 29 entries 28'10 (0'0) error 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head by client.4488.0:1154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'8)", 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:26.976 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87873247099999996, 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051115789999999998, 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:26.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053160+0000", 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053167+0000", 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6129999999999998e-06 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053193+0000", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5592000000000002e-05 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053208+0000", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5213e-05 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053304+0000", 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5874999999999999e-05 2024-09-17T22:57:26.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058271+0000", 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049666909999999996 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1159 3.3 3:c56ebdd4:::rbd_data.118840f89003.00000000000000b2:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87862377300000005, 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050648550000000001, 2024-09-17T22:57:26.980 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1159 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:26.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053269+0000", 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3300000000000001e-07 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053273+0000", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053282+0000", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.764e-06 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053299+0000", 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7108999999999998e-05 2024-09-17T22:57:26.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053348+0000", 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8436999999999997e-05 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053465+0000", 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011670100000000001 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058333+0000", 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048680249999999998 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1177 3.3 3:c6fe237e:::rbd_data.118840f89003.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87538767299999998, 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051796150000000003, 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.985 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1177 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056505+0000", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0449999999999999e-06 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056510+0000", 2024-09-17T22:57:26.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056522+0000", 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1457000000000001e-05 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056529+0000", 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056584+0000", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5813999999999998e-05 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056667+0000", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2515999999999994e-05 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056824+0000", 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157331 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061278+0000", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044539879999999999 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061600+0000", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032172 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061656+0000", 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5662999999999999e-05 2024-09-17T22:57:26.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061683+0000", 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7895e-05 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1173 3.a e28/22 3:5642bcd0:::rbd_data.118840f89003.00000000000000af:head v 28'16, mlcod=28'14)", 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87516915500000003, 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045820289999999996, 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056723+0000", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.68e-07 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056729+0000", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056739+0000", 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0200000000000001e-05 2024-09-17T22:57:26.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056746+0000", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2099999999999996e-06 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056792+0000", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5729999999999998e-05 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056807+0000", 2024-09-17T22:57:26.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5017999999999999e-05 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061265+0000", 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044575509999999997 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061304+0000", 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9888999999999997e-05 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:26.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1186 3.3 3:c56ebdd4:::rbd_data.118840f89003.00000000000000b2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87298021599999998, 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053064840000000002, 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1186 2024-09-17T22:57:26.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:26.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058912+0000", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000001e-07 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058916+0000", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1570000000000002e-06 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058926+0000", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:26.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058931+0000", 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0329999999999999e-06 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058974+0000", 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3426000000000001e-05 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059022+0000", 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8344999999999999e-05 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059136+0000", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113418 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063873+0000", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047369480000000004 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064085+0000", 2024-09-17T22:57:27.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021247099999999999 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064174+0000", 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9042999999999994e-05 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064218+0000", 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3565e-05 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1187 3.1c e28/22 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head v 28'11, mlcod=28'9)", 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.87251219099999999, 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045253089999999999, 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059380+0000", 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059385+0000", 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.366e-06 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059397+0000", 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1739999999999999e-05 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059404+0000", 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.657e-06 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059440+0000", 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6727999999999998e-05 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059451+0000", 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0458e-05 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063877+0000", 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044256410000000001 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063905+0000", 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8073999999999999e-05 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1199 3.3 3:c7a091de:::rbd_data.118840f89003.00000000000000ba:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.86990303000000002, 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049477330000000002, 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1199 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:27.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6899999999999996e-07 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061995+0000", 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7200000000000003e-06 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062009+0000", 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4287000000000001e-05 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062017+0000", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6029999999999999e-06 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062067+0000", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9866e-05 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062172+0000", 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000105028 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066936+0000", 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047642600000000002 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1217 3.3 3:c7a091de:::rbd_data.118840f89003.00000000000000ba:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.86437132299999997, 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056857569999999996, 2024-09-17T22:57:27.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1217 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067521+0000", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067527+0000", 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2709999999999998e-06 2024-09-17T22:57:27.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067539+0000", 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1516e-05 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067547+0000", 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8550000000000002e-06 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067623+0000", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6631000000000005e-05 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067705+0000", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1144999999999996e-05 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067899+0000", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000194441 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.072399+0000", 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044999630000000001 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073110+0000", 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00071130800000000001 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073166+0000", 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5269999999999998e-05 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073206+0000", 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0398999999999997e-05 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1223 3.3 3:c711af7a:::rbd_data.118840f89003.00000000000000c3:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.86203759400000002, 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052997110000000004, 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1223 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069855+0000", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6000000000000003e-07 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069858+0000", 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7170000000000002e-06 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069869+0000", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0356e-05 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069874+0000", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069918+0000", 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4320000000000003e-05 2024-09-17T22:57:27.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.070035+0000", 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117278 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075154+0000", 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051184769999999997 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.a epoch 28/22 rep_tid 29 entries 28'17 (0'0) error 3:562220c4:::rbd_data.118840f89003.00000000000000c4:head by client.4488.0:1233 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'15)", 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.85810983200000002, 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049116289999999998, 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073789+0000", 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:57:27.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073808+0000", 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8649000000000001e-05 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073824+0000", 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5566999999999999e-05 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073884+0000", 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0748999999999997e-05 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.078693+0000", 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048088949999999997 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1241 3.3 3:c711af7a:::rbd_data.118840f89003.00000000000000c3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:27.025 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.85619041299999998, 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004755296, 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1241 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075702+0000", 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9100000000000003e-07 2024-09-17T22:57:27.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075711+0000", 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1390000000000001e-06 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075721+0000", 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.029e-05 2024-09-17T22:57:27.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075726+0000", 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075769+0000", 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2805000000000001e-05 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075832+0000", 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.279e-05 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075980+0000", 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014830099999999999 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079852+0000", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003871846 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080356+0000", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050418700000000002 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080413+0000", 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6645999999999999e-05 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080456+0000", 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3591999999999999e-05 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1254 3.1c e28/22 3:3e41df91:::rbd_data.118840f89003.00000000000000cb:head v 28'13, mlcod=28'10)", 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:27.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.85091177500000004, 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060906620000000002, 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:27.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3799999999999997e-07 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080993+0000", 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2381000000000001e-05 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081003+0000", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081009+0000", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4969999999999998e-06 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081063+0000", 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3591999999999998e-05 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081085+0000", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2098000000000001e-05 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087031+0000", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059464030000000003 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087070+0000", 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9214000000000001e-05 2024-09-17T22:57:27.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.037 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:27.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:57:27.039 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:27.039 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:27.036+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:27.045 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T22:57:27.139 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:27.136+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:27.183 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:27.184 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:27.192 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:27.240 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:27.237+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:27.281 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:27.326 INFO:teuthology.orchestra.run.smithi159.stdout:173 write 0xbe6d60e thru 0xbe7d329 (0xfd1c bytes) 2024-09-17T22:57:27.329 INFO:teuthology.orchestra.run.smithi159.stdout:175 read 0x63b51a4 thru 0x63b62cc (0x1129 bytes) 2024-09-17T22:57:27.329 INFO:teuthology.orchestra.run.smithi159.stdout:176 read 0xca7751 thru 0xca8f41 (0x17f1 bytes) 2024-09-17T22:57:27.330 INFO:teuthology.orchestra.run.smithi159.stdout:177 punch from 0x1bfcf7a to 0x1c01b4a, (0x4bd0 bytes) 2024-09-17T22:57:27.330 INFO:teuthology.orchestra.run.smithi159.stdout:178 read 0x14082e1 thru 0x140f17f (0x6e9f bytes) 2024-09-17T22:57:27.330 INFO:teuthology.orchestra.run.smithi159.stdout:179 writesame 0x299915 thru 0x2a1880 data_size 0x46(0x7f6c bytes) 2024-09-17T22:57:27.342 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:27.342 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:27.343 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:27.343 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:180 write 0xb9fe704 thru 0xba0423e (0x5b3b bytes) 2024-09-17T22:57:27.343 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:27.340+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:27.344 INFO:teuthology.orchestra.run.smithi159.stdout:181 punch from 0x3255020 to 0x3259b21, (0x4b01 bytes) 2024-09-17T22:57:27.344 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.344 INFO:teuthology.orchestra.run.smithi159.stdout:183 read 0x92fadae thru 0x93078a5 (0xcaf8 bytes) 2024-09-17T22:57:27.344 INFO:teuthology.orchestra.run.smithi159.stdout:184 read 0x9061da5 thru 0x906c8d9 (0xab35 bytes) 2024-09-17T22:57:27.344 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0xe80112f 2024-09-17T22:57:27.346 INFO:teuthology.orchestra.run.smithi159.stdout:185 trunc from 0xbe7d32a to 0xe80112f 2024-09-17T22:57:27.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:27.352 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.353 INFO:teuthology.orchestra.run.smithi159.stdout:190 write 0xd483523 thru 0xd485b4a (0x2628 bytes) 2024-09-17T22:57:27.355 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.356 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.356 INFO:teuthology.orchestra.run.smithi159.stdout:193 punch from 0x5b5d550 to 0x5b5fa92, (0x2542 bytes) 2024-09-17T22:57:27.377 INFO:teuthology.orchestra.run.smithi159.stdout:195 trunc from 0xe80112f to 0x270cce 2024-09-17T22:57:27.384 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.385 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.385 INFO:teuthology.orchestra.run.smithi159.stdout:199 read 0x132a47 thru 0x13c0f9 (0x96b3 bytes) 2024-09-17T22:57:27.385 INFO:teuthology.orchestra.run.smithi159.stdout:200 read 0x17169 thru 0x1caf3 (0x598b bytes) 2024-09-17T22:57:27.406 INFO:teuthology.orchestra.run.smithi159.stdout:202 writesame 0xcdb1ed2 thru 0xcdb4456 data_size 0x11(0x2585 bytes) 2024-09-17T22:57:27.406 INFO:teuthology.orchestra.run.smithi159.stdout:204 punch from 0x9ed14a to 0x9f8ee4, (0xbd9a bytes) 2024-09-17T22:57:27.406 INFO:teuthology.orchestra.run.smithi159.stdout:205 write 0xfb02b9 thru 0xfb7891 (0x75d9 bytes) 2024-09-17T22:57:27.407 INFO:teuthology.orchestra.run.smithi159.stdout:207 writesame 0x5c0b3ab thru 0x5c16219 data_size 0x41(0xae6f bytes) 2024-09-17T22:57:27.410 INFO:teuthology.orchestra.run.smithi159.stdout:208 read 0x8637f3d thru 0x86450f9 (0xd1bd bytes) 2024-09-17T22:57:27.411 INFO:teuthology.orchestra.run.smithi159.stdout:209 writesame 0x3adc850 thru 0x3ae15e2 data_size 0x1(0x4d93 bytes) 2024-09-17T22:57:27.422 INFO:teuthology.orchestra.run.smithi159.stdout:211 write 0x25e2db2 thru 0x25eae38 (0x8087 bytes) 2024-09-17T22:57:27.423 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:27.423 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:27.430 INFO:teuthology.orchestra.run.smithi159.stdout:212 trunc from 0xcdb4457 to 0x90cbdb1 2024-09-17T22:57:27.435 INFO:teuthology.orchestra.run.smithi159.stdout:213 punch from 0x786ae30 to 0x786f65d, (0x482d bytes) 2024-09-17T22:57:27.438 INFO:teuthology.orchestra.run.smithi159.stdout:217 trunc from 0x90cbdb1 to 0xb6d2d70 2024-09-17T22:57:27.443 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:27.443 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:27.443 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.444 INFO:teuthology.orchestra.run.smithi159.stdout:219 punch from 0x55ebf6d to 0x55faa20, (0xeab3 bytes) 2024-09-17T22:57:27.444 INFO:teuthology.orchestra.run.smithi159.stdout:220 write 0xb181a90 thru 0xb190393 (0xe904 bytes) 2024-09-17T22:57:27.444 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:27.441+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:27.449 INFO:teuthology.orchestra.run.smithi159.stdout:221 trunc from 0xb6d2d70 to 0xc223f86 2024-09-17T22:57:27.455 INFO:teuthology.orchestra.run.smithi159.stdout:222 read 0x42b8f2f thru 0x42c1797 (0x8869 bytes) 2024-09-17T22:57:27.455 INFO:teuthology.orchestra.run.smithi159.stdout:224 write 0x2d15b57 thru 0x2d1c3cd (0x6877 bytes) 2024-09-17T22:57:27.458 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.458 INFO:teuthology.orchestra.run.smithi159.stdout:226 writesame 0x1ee7f8 thru 0x1fcfb3 data_size 0x2(0xe7bc bytes) 2024-09-17T22:57:27.461 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:27.461 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:27.474 INFO:teuthology.orchestra.run.smithi159.stdout:227 read 0x4063fa1 thru 0x406f75e (0xb7be bytes) 2024-09-17T22:57:27.474 INFO:teuthology.orchestra.run.smithi159.stdout:230 read 0x11a7e6 thru 0x127c6a (0xd485 bytes) 2024-09-17T22:57:27.474 INFO:teuthology.orchestra.run.smithi159.stdout:231 write 0x82f7756 thru 0x82f7a51 (0x2fc bytes) 2024-09-17T22:57:27.481 INFO:teuthology.orchestra.run.smithi159.stdout:232 trunc from 0xc223f86 to 0x89ab697 2024-09-17T22:57:27.482 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:27.482 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:27.493 INFO:teuthology.orchestra.run.smithi159.stdout:233 read 0x23c9752 thru 0x23cfca3 (0x6552 bytes) 2024-09-17T22:57:27.500 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:27.500 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4275.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:09.885275+0000", 2024-09-17T22:57:27.503 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.614634688999999, 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010877421, 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4275", 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/3299563960", 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885275+0000", 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885275+0000", 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885281+0000", 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.57e-06 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885295+0000", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4362000000000001e-05 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885311+0000", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5472999999999999e-05 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885327+0000", 2024-09-17T22:57:27.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6350000000000001e-05 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885650+0000", 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032297700000000001 2024-09-17T22:57:27.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.885660+0000", 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0121e-05 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.890426+0000", 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047661530000000004 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.890603+0000", 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017709099999999999 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.890757+0000", 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015340500000000001 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.892432+0000", 2024-09-17T22:57:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016748780000000001 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.896088+0000", 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003656285 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.896122+0000", 2024-09-17T22:57:27.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4446000000000002e-05 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:09.896153+0000", 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0309999999999999e-05 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4275.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:11.897927+0000", 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.601983435999999, 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041846890000000001, 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4275", 2024-09-17T22:57:27.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/3299563960", 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897927+0000", 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897927+0000", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897929+0000", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9299999999999999e-06 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897940+0000", 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0988999999999999e-05 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897960+0000", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9687e-05 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.897969+0000", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9949999999999994e-06 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.898047+0000", 2024-09-17T22:57:27.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7718000000000004e-05 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.898054+0000", 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.277e-06 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.899745+0000", 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016906169999999999 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.899812+0000", 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7601999999999994e-05 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.899915+0000", 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102907 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.901240+0000", 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013248520000000001 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.902042+0000", 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000801843 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.902080+0000", 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8018000000000001e-05 2024-09-17T22:57:27.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:11.902111+0000", 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1254000000000003e-05 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:11 3.1f e25/22 3:fa654f18:::rbd_id.image_client.0:head v 25'1, mlcod=0'0)", 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:21.855359+0000", 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.64455098, 2024-09-17T22:57:27.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044452650000000003, 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855359+0000", 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855359+0000", 2024-09-17T22:57:27.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855359+0000", 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855363+0000", 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6669999999999998e-06 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855378+0000", 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4951e-05 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855384+0000", 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0859999999999998e-06 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855430+0000", 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5884000000000003e-05 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.855447+0000", 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6671e-05 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.859772+0000", 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043253559999999998 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:21.859804+0000", 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2369000000000001e-05 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:739 3.1c 3:3e41df91:::rbd_data.118840f89003.00000000000000cb:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T22:57:27.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:25.349734+0000", 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.1501761159999999, 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0092345250000000004, 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.525 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 739 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349734+0000", 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349734+0000", 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349735+0000", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8899999999999998e-07 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349740+0000", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4319999999999998e-06 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349751+0000", 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0920999999999999e-05 2024-09-17T22:57:27.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349756+0000", 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9860000000000002e-06 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349803+0000", 2024-09-17T22:57:27.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7023e-05 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.349885+0000", 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2064000000000006e-05 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.351296+0000", 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014110660000000001 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.358890+0000", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0075933950000000002 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.358901+0000", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1739000000000001e-05 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.358933+0000", 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829000000000003e-05 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:25.358968+0000", 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5281000000000003e-05 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1012 3.c 3:345d9f7d:::rbd_data.118840f89003.0000000000000089:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.024335+0000", 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.475575004, 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047481479999999998, 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1012 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024335+0000", 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024335+0000", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024336+0000", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4300000000000003e-07 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024339+0000", 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3419999999999999e-06 2024-09-17T22:57:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024345+0000", 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3409999999999999e-06 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024351+0000", 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4369999999999998e-06 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.024397+0000", 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6291000000000001e-05 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.025208+0000", 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00081142599999999997 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.029083+0000", 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038746679999999999 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1046 3.5 3:a332fa3f:::rbd_data.118840f89003.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.028058+0000", 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4718521070000001, 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043728559999999996, 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1046 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028058+0000", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028058+0000", 2024-09-17T22:57:27.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028059+0000", 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028064+0000", 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5140000000000001e-06 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028078+0000", 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3805000000000001e-05 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028096+0000", 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7739000000000001e-05 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028174+0000", 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8015000000000004e-05 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028252+0000", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8092999999999999e-05 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028432+0000", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000180087 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.029153+0000", 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072037900000000005 2024-09-17T22:57:27.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032330+0000", 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0031776090000000001 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032387+0000", 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6818999999999999e-05 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032431+0000", 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3806e-05 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1102 3.1d 3:b8b32f6b:::rbd_data.118840f89003.000000000000009e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.038084+0000", 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.461825524, 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005694859, 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.542 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1102 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038084+0000", 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038084+0000", 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038085+0000", 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-06 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038091+0000", 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4299999999999997e-06 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038104+0000", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2842e-05 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038111+0000", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038169+0000", 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8557999999999997e-05 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038229+0000", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9596000000000003e-05 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038349+0000", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011984700000000001 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.040639+0000", 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00229014 2024-09-17T22:57:27.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043714+0000", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0030752230000000002 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043746+0000", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1795000000000003e-05 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043779+0000", 2024-09-17T22:57:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3312999999999998e-05 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1154 3.1c 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.052821+0000", 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4470894919999999, 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056135969999999997, 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1154 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052821+0000", 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052821+0000", 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052821+0000", 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9899999999999999e-07 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052825+0000", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-06 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052836+0000", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0383999999999999e-05 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052848+0000", 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1664e-05 2024-09-17T22:57:27.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052892+0000", 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4490999999999998e-05 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.052976+0000", 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4091999999999994e-05 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058434+0000", 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005457997 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1165 3.17 e28/22 3:ebdd7e77:::rbd_data.118840f89003.00000000000000ac:head v 28'5, mlcod=25'3)", 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.054030+0000", 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.445880378, 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041753820000000001, 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.553 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054030+0000", 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054030+0000", 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054030+0000", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9699999999999995e-07 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054038+0000", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054048+0000", 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.91e-06 2024-09-17T22:57:27.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054052+0000", 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5299999999999998e-06 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054095+0000", 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2899999999999999e-05 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.054112+0000", 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7048e-05 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058165+0000", 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040527469999999998 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058205+0000", 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0265000000000002e-05 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.557 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1187 3.1c 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.059083+0000", 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.440827021, 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050166289999999999, 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1187 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059083+0000", 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059083+0000", 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059084+0000", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4900000000000005e-07 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059088+0000", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6299999999999997e-06 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059099+0000", 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.079e-05 2024-09-17T22:57:27.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059106+0000", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5799999999999997e-06 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059148+0000", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1708000000000001e-05 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059197+0000", 2024-09-17T22:57:27.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9465999999999997e-05 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059322+0000", 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012454 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061143+0000", 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018218030000000001 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064008+0000", 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002865032 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064064+0000", 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5444e-05 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064100+0000", 2024-09-17T22:57:27.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5787000000000001e-05 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1206 3.11 3:8eee0039:::rbd_data.118840f89003.00000000000000bc:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.435582664, 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90939317600000003, 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1206 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:27.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064328+0000", 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6899999999999997e-07 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064331+0000", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0589999999999998e-06 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064338+0000", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064342+0000", 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:57:27.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064385+0000", 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2216999999999999e-05 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064391+0000", 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.117e-06 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972129+0000", 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90773784000000002 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972245+0000", 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116396 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973721+0000", 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001475456 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1210 3.11 3:8af04179:::rbd_data.118840f89003.00000000000000bf:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.43293023, 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90700086899999999, 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1210 2024-09-17T22:57:27.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066984+0000", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7249999999999999e-06 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066996+0000", 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2337000000000001e-05 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067002+0000", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6420000000000004e-06 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067055+0000", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2797000000000003e-05 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067064+0000", 2024-09-17T22:57:27.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0569999999999999e-06 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972638+0000", 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90557432699999996 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972737+0000", 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8486999999999994e-05 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973981+0000", 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012439720000000001 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1218 3.16 3:6a4cb7f8:::rbd_data.118840f89003.00000000000000bb:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.068127+0000", 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.431782943, 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052928130000000004, 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1218 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068127+0000", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068127+0000", 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068128+0000", 2024-09-17T22:57:27.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:57:27.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068132+0000", 2024-09-17T22:57:27.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5770000000000004e-06 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068142+0000", 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0043e-05 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068149+0000", 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068186+0000", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7441999999999998e-05 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068242+0000", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5980000000000003e-05 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.068363+0000", 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000121407 2024-09-17T22:57:27.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.070696+0000", 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023329650000000002 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073294+0000", 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025972119999999998 2024-09-17T22:57:27.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073377+0000", 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3496000000000002e-05 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073420+0000", 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2753000000000002e-05 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.584 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1235 3.11 3:8a065d9f:::rbd_data.118840f89003.00000000000000c6:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.426418532, 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90162610200000004, 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.585 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1235 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073492+0000", 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073497+0000", 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8099999999999997e-06 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073510+0000", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073517+0000", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0389999999999999e-06 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073571+0000", 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4159000000000001e-05 2024-09-17T22:57:27.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073580+0000", 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0880000000000001e-06 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972930+0000", 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89935040799999999 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973012+0000", 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1601999999999996e-05 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975118+0000", 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021056640000000001 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.590 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1242 3.c 3:35b7b88a:::rbd_data.118840f89003.00000000000000c7:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.076431+0000", 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4234792030000001, 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039042180000000001, 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1242 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076431+0000", 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076431+0000", 2024-09-17T22:57:27.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076432+0000", 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076435+0000", 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9890000000000003e-06 2024-09-17T22:57:27.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076445+0000", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9040000000000004e-06 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076450+0000", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076492+0000", 2024-09-17T22:57:27.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1990000000000003e-05 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.076576+0000", 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3220000000000006e-05 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080335+0000", 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003759419 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1249 3.11 3:89eed2ba:::rbd_data.118840f89003.00000000000000c8:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.419979122, 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89530808699999997, 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1249 2024-09-17T22:57:27.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079932+0000", 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5000000000000002e-07 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079941+0000", 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7119999999999995e-06 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079953+0000", 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1841000000000001e-05 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079961+0000", 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5599999999999996e-06 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079999+0000", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8642e-05 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080005+0000", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2029999999999997e-06 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973323+0000", 2024-09-17T22:57:27.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89331756399999995 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973409+0000", 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5636999999999993e-05 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975239+0000", 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018302780000000001 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1254 3.1c 3:3e41df91:::rbd_data.118840f89003.00000000000000cb:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.080580+0000", 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4193298429999999, 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0066886009999999997, 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1254 2024-09-17T22:57:27.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080580+0000", 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080580+0000", 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080581+0000", 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080585+0000", 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1710000000000004e-06 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080597+0000", 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089000000000001e-05 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080605+0000", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4460000000000004e-06 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080659+0000", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4407999999999998e-05 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080742+0000", 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3400999999999997e-05 2024-09-17T22:57:27.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080917+0000", 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017463999999999999 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.084499+0000", 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00358224 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087176+0000", 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002676753 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087227+0000", 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0861e-05 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087269+0000", 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1829000000000002e-05 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1256 3.c 3:35b7b88a:::rbd_data.118840f89003.00000000000000c7:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.080905+0000", 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4190051779999999, 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048237610000000002, 2024-09-17T22:57:27.608 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1256 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080905+0000", 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080905+0000", 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080906+0000", 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0500000000000002e-07 2024-09-17T22:57:27.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080910+0000", 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2089999999999999e-06 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080920+0000", 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7100000000000002e-06 2024-09-17T22:57:27.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080930+0000", 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080961+0000", 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1887999999999997e-05 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081007+0000", 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5875999999999999e-05 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081195+0000", 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018723599999999999 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081696+0000", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050126499999999998 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.085664+0000", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0039680430000000001 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.085697+0000", 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3053999999999999e-05 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.085729+0000", 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1646000000000002e-05 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1259 3.c 3:30186e42:::rbd_data.118840f89003.00000000000000cd:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.083204+0000", 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4167057810000001, 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052729969999999998, 2024-09-17T22:57:27.615 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1259 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083204+0000", 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083204+0000", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083206+0000", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.336e-06 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083213+0000", 2024-09-17T22:57:27.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.063e-06 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083228+0000", 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5163e-05 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083237+0000", 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3130000000000004e-06 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083318+0000", 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0846000000000005e-05 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.083411+0000", 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2635999999999999e-05 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.088477+0000", 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050666399999999999 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1292 3.16 3:6ad3dd83:::rbd_data.118840f89003.00000000000000d4:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.094430+0000", 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.4054799440000001, 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0040708599999999999, 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:27.620 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1292 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094430+0000", 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094430+0000", 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094431+0000", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9299999999999996e-07 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094436+0000", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6369999999999998e-06 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094450+0000", 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4671000000000001e-05 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094458+0000", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3010000000000001e-06 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094535+0000", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7117e-05 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.094639+0000", 2024-09-17T22:57:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104808 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.098501+0000", 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038614330000000001 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:27.624 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:27.626 INFO:teuthology.orchestra.run.smithi159.stdout:236 write 0xc8aad0f thru 0xc8ace1a (0x210c bytes) 2024-09-17T22:57:27.626 INFO:teuthology.orchestra.run.smithi159.stdout:240 write 0x5fa6287 thru 0x5fb2733 (0xc4ad bytes) 2024-09-17T22:57:27.626 INFO:teuthology.orchestra.run.smithi159.stdout:241 punch from 0xba95f6a to 0xba9bcde, (0x5d74 bytes) 2024-09-17T22:57:27.626 INFO:teuthology.orchestra.run.smithi159.stdout:243 read 0x16c9dd2 thru 0x16d6dca (0xcff9 bytes) 2024-09-17T22:57:27.627 INFO:teuthology.orchestra.run.smithi159.stdout:244 trunc from 0xc8ace1b to 0xbfcb62c 2024-09-17T22:57:27.627 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:27.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:27.628 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:27.624+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:27.628 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:246 trunc from 0xbfcb62c to 0xca4c76c 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:248 write 0x7b0a8b7 thru 0x7b0aaa1 (0x1eb bytes) 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:249 punch from 0xbeb2c66 to 0xbebaaf8, (0x7e92 bytes) 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:251 read 0x78caa9b thru 0x78d564b (0xabb1 bytes) 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:253 writesame 0x9cf7958 thru 0x9cf8ff6 data_size 0x1(0x169f bytes) 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:27.629 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:27.630 INFO:teuthology.orchestra.run.smithi159.stdout:259 write 0xd084c44 thru 0xd093090 (0xe44d bytes) 2024-09-17T22:57:27.630 INFO:teuthology.orchestra.run.smithi159.stdout:262 write 0x82167f thru 0x82da82 (0xc404 bytes) 2024-09-17T22:57:27.630 INFO:teuthology.orchestra.run.smithi159.stdout:264 read 0x2a84c48 thru 0x2a8bf16 (0x72cf bytes) 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stdout:265 read 0x8f6472 thru 0x8fbf27 (0x5ab6 bytes) 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stdout:268 read 0x574af06 thru 0x5756ad3 (0xbbce bytes) 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stdout:269 write 0xdc1d66c thru 0xdc2bf52 (0xe8e7 bytes) 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:27.631 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:272 write 0x52a28ba thru 0x52ad6de (0xae25 bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:273 read 0x81078ae thru 0x8113a1a (0xc16d bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:275 read 0x5259c79 thru 0x5268e88 (0xf210 bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:276 punch from 0xba6d8ea to 0xba7ab8e, (0xd2a4 bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:277 read 0x3b86f75 thru 0x3b899dd (0x2a69 bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:278 write 0xd294fe3 thru 0xd2a3c57 (0xec75 bytes) 2024-09-17T22:57:27.632 INFO:teuthology.orchestra.run.smithi159.stdout:280 write 0x66454a6 thru 0x6647ce1 (0x283c bytes) 2024-09-17T22:57:27.634 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:27.634 INFO:teuthology.orchestra.run.smithi159.stdout:284 write 0xe07aedf thru 0xe082777 (0x7899 bytes) 2024-09-17T22:57:27.634 INFO:teuthology.orchestra.run.smithi159.stdout:285 read 0x13eadf5 thru 0x13f8c82 (0xde8e bytes) 2024-09-17T22:57:27.634 INFO:teuthology.orchestra.run.smithi159.stdout:287 clone 4 order 19 su 262144 sc 14 2024-09-17T22:57:27.726 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:27.724+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:27.771 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:27.771 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:27.780 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:27.827 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:27.825+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:27.885 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:27.886 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:27.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:27.935 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:27.933+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:27.974 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:57:28.030 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:28.028+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:28.039 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone3 from 0xe082778 (overlap 0x270cce) to 0x23251f 2024-09-17T22:57:28.045 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.045 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.056 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:28.112 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T22:57:28.130 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:28.128+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:28.166 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.166 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.174 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:28.175 INFO:teuthology.orchestra.run.smithi159.stdout:288 write 0x5cfc98a thru 0x5d08b9a (0xc211 bytes) 2024-09-17T22:57:28.184 INFO:teuthology.orchestra.run.smithi159.stdout:289 flatten 2024-09-17T22:57:28.231 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:28.229+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:28.238 INFO:teuthology.orchestra.run.smithi159.stdout:290 punch from 0x1c29b74 to 0x1c36504, (0xc990 bytes) 2024-09-17T22:57:28.245 INFO:teuthology.orchestra.run.smithi159.stdout:291 trunc from 0xe082778 to 0x7e2b0ab 2024-09-17T22:57:28.261 INFO:teuthology.orchestra.run.smithi159.stdout:292 trunc from 0x7e2b0ab to 0x723dc14 2024-09-17T22:57:28.271 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:28.271 INFO:teuthology.orchestra.run.smithi159.stdout:294 read 0x3680a8b thru 0x368e877 (0xdded bytes) 2024-09-17T22:57:28.271 INFO:teuthology.orchestra.run.smithi159.stdout:296 write 0x4daa6be thru 0x4dad022 (0x2965 bytes) 2024-09-17T22:57:28.273 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:28.289 INFO:teuthology.orchestra.run.smithi159.stdout:299 writesame 0xec69397 thru 0xec6f2fd data_size 0x7(0x5f67 bytes) 2024-09-17T22:57:28.290 INFO:teuthology.orchestra.run.smithi159.stdout:300 writesame 0xb25ce09 thru 0xb265fd1 data_size 0x1(0x91c9 bytes) 2024-09-17T22:57:28.291 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.291 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.300 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:28.309 INFO:teuthology.orchestra.run.smithi159.stdout:301 write 0x8af1ad3 thru 0x8af8c89 (0x71b7 bytes) 2024-09-17T22:57:28.318 INFO:teuthology.orchestra.run.smithi159.stdout:303 trunc from 0xec6f2fe to 0xa2c95df 2024-09-17T22:57:28.328 INFO:teuthology.orchestra.run.smithi159.stdout:304 punch from 0x3272fc9 to 0x3273634, (0x66b bytes) 2024-09-17T22:57:28.328 INFO:teuthology.orchestra.run.smithi159.stdout:305 write 0x565eec4 thru 0x5662080 (0x31bd bytes) 2024-09-17T22:57:28.334 INFO:teuthology.orchestra.run.smithi159.stdout:306 trunc from 0xa2c95df to 0xce6e1ab 2024-09-17T22:57:28.340 INFO:teuthology.orchestra.run.smithi159.stdout:308 read 0x3548721 thru 0x3555733 (0xd013 bytes) 2024-09-17T22:57:28.340 INFO:teuthology.orchestra.run.smithi159.stdout:309 read 0x99b802f thru 0x99c4479 (0xc44b bytes) 2024-09-17T22:57:28.340 INFO:teuthology.orchestra.run.smithi159.stdout:310 punch from 0x91b1993 to 0x91bce7b, (0xb4e8 bytes) 2024-09-17T22:57:28.344 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:28.342+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:28.346 INFO:teuthology.orchestra.run.smithi159.stdout:311 writesame 0xdaf835c thru 0xdafc026 data_size 0x1(0x3ccb bytes) 2024-09-17T22:57:28.362 INFO:teuthology.orchestra.run.smithi159.stdout:312 trunc from 0xdafc027 to 0x1170a1a 2024-09-17T22:57:28.393 INFO:teuthology.orchestra.run.smithi159.stdout:316 write 0xb100dfb thru 0xb10ccfd (0xbf03 bytes) 2024-09-17T22:57:28.396 INFO:teuthology.orchestra.run.smithi159.stdout:319 writesame 0xac8b255 thru 0xac8eefc data_size 0xc(0x3ca8 bytes) 2024-09-17T22:57:28.400 INFO:teuthology.orchestra.run.smithi159.stdout:320 write 0x57d9be1 thru 0x57e01f0 (0x6610 bytes) 2024-09-17T22:57:28.402 INFO:teuthology.orchestra.run.smithi159.stdout:321 writesame 0x315685c thru 0x3162885 data_size 0x1(0xc02a bytes) 2024-09-17T22:57:28.408 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.408 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:28.427 INFO:teuthology.orchestra.run.smithi159.stdout:322 read 0x1af34f7 thru 0x1afd46c (0x9f76 bytes) 2024-09-17T22:57:28.434 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:28.431+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:28.438 INFO:teuthology.orchestra.run.smithi159.stdout:323 trunc from 0xb10ccfe to 0x1aaad52 2024-09-17T22:57:28.464 INFO:teuthology.orchestra.run.smithi159.stdout:327 trunc from 0x1aaad52 to 0xc0b8930 2024-09-17T22:57:28.470 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:28.470 INFO:teuthology.orchestra.run.smithi159.stdout:333 write 0xb4b5b64 thru 0xb4baaf7 (0x4f94 bytes) 2024-09-17T22:57:28.473 INFO:teuthology.orchestra.run.smithi159.stdout:334 read 0x1f6930b thru 0x1f6c100 (0x2df6 bytes) 2024-09-17T22:57:28.480 INFO:teuthology.orchestra.run.smithi159.stdout:335 writesame 0xd66417f thru 0xd668845 data_size 0x1(0x46c7 bytes) 2024-09-17T22:57:28.480 INFO:teuthology.orchestra.run.smithi159.stdout:336 clone 5 order 24 su 4194304 sc 2 2024-09-17T22:57:28.530 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.531 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.534 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:28.531+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:28.539 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:28.635 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:28.633+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:28.682 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.682 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:28.737 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:28.734+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:28.794 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.794 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.803 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:28.847 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:28.845+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:28.906 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:28.907 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:28.916 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:28.958 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:28.955+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:29.039 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:29.037+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:29.093 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.094 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.106 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:29.141 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:29.138+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:29.215 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T22:57:29.221 INFO:teuthology.orchestra.run.smithi159.stdout:337 read 0x708a0e7 thru 0x708c9cc (0x28e6 bytes) 2024-09-17T22:57:29.221 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.221 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.232 INFO:teuthology.orchestra.run.smithi159.stdout:344 trunc from 0xd668846 to 0x4bdeaa9 2024-09-17T22:57:29.242 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:29.239+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:29.261 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.261 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.261 INFO:teuthology.orchestra.run.smithi159.stdout:345 write 0xb79b71f thru 0xb79c245 (0xb27 bytes) 2024-09-17T22:57:29.265 INFO:teuthology.orchestra.run.smithi159.stdout:346 write 0x392d7e0 thru 0x393425e (0x6a7f bytes) 2024-09-17T22:57:29.269 INFO:teuthology.orchestra.run.smithi159.stdout:347 write 0x3b9c480 thru 0x3ba6cd9 (0xa85a bytes) 2024-09-17T22:57:29.270 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:29.273 INFO:teuthology.orchestra.run.smithi159.stdout:348 trunc from 0xb79c246 to 0xd3ffdf6 2024-09-17T22:57:29.278 INFO:teuthology.orchestra.run.smithi159.stdout:349 write 0x635b423 thru 0x636a462 (0xf040 bytes) 2024-09-17T22:57:29.282 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0xeb9d89d 2024-09-17T22:57:29.284 INFO:teuthology.orchestra.run.smithi159.stdout:350 trunc from 0xd3ffdf6 to 0xeb9d89d 2024-09-17T22:57:29.289 INFO:teuthology.orchestra.run.smithi159.stdout:352 write 0x6bc6edd thru 0x6bd34b4 (0xc5d8 bytes) 2024-09-17T22:57:29.291 INFO:teuthology.orchestra.run.smithi159.stdout:355 read 0x9a0d990 thru 0x9a11718 (0x3d89 bytes) 2024-09-17T22:57:29.291 INFO:teuthology.orchestra.run.smithi159.stdout:356 punch from 0xf3293a to 0xf3d1a2, (0xa868 bytes) 2024-09-17T22:57:29.291 INFO:teuthology.orchestra.run.smithi159.stdout:358 write 0x37dbaa thru 0x38b261 (0xd6b8 bytes) 2024-09-17T22:57:29.296 INFO:teuthology.orchestra.run.smithi159.stdout:359 write 0x81b96cf thru 0x81bea43 (0x5375 bytes) 2024-09-17T22:57:29.299 INFO:teuthology.orchestra.run.smithi159.stdout:361 write 0x3831602 thru 0x3834bb3 (0x35b2 bytes) 2024-09-17T22:57:29.300 INFO:teuthology.orchestra.run.smithi159.stdout:362 write 0x5591327 thru 0x559ecd9 (0xd9b3 bytes) 2024-09-17T22:57:29.303 INFO:teuthology.orchestra.run.smithi159.stdout:363 write 0x2212745 thru 0x22211c1 (0xea7d bytes) 2024-09-17T22:57:29.305 INFO:teuthology.orchestra.run.smithi159.stdout:364 read 0x5512194 thru 0x551d3e7 (0xb254 bytes) 2024-09-17T22:57:29.306 INFO:teuthology.orchestra.run.smithi159.stdout:365 read 0xdad6a0f thru 0xdae4925 (0xdf17 bytes) 2024-09-17T22:57:29.306 INFO:teuthology.orchestra.run.smithi159.stdout:366 write 0xb96728f thru 0xb971394 (0xa106 bytes) 2024-09-17T22:57:29.309 INFO:teuthology.orchestra.run.smithi159.stdout:368 write 0x67a9b99 thru 0x67b3df3 (0xa25b bytes) 2024-09-17T22:57:29.312 INFO:teuthology.orchestra.run.smithi159.stdout:369 punch from 0xbb2a64b to 0xbb2d5ac, (0x2f61 bytes) 2024-09-17T22:57:29.313 INFO:teuthology.orchestra.run.smithi159.stdout:371 write 0x698e107 thru 0x699372e (0x5628 bytes) 2024-09-17T22:57:29.314 INFO:teuthology.orchestra.run.smithi159.stdout:372 punch from 0x96cac66 to 0x96d181c, (0x6bb6 bytes) 2024-09-17T22:57:29.314 INFO:teuthology.orchestra.run.smithi159.stdout:373 read 0xabc321c thru 0xabd1667 (0xe44c bytes) 2024-09-17T22:57:29.314 INFO:teuthology.orchestra.run.smithi159.stdout:374 write 0x7fe7be1 thru 0x7fe7d89 (0x1a9 bytes) 2024-09-17T22:57:29.316 INFO:teuthology.orchestra.run.smithi159.stdout:375 writesame 0x9663368 thru 0x9669501 data_size 0x1f(0x619a bytes) 2024-09-17T22:57:29.320 INFO:teuthology.orchestra.run.smithi159.stdout:376 read 0x92ef7fb thru 0x92f89ea (0x91f0 bytes) 2024-09-17T22:57:29.320 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.320 INFO:teuthology.orchestra.run.smithi159.stdout:379 read 0x604816b thru 0x60551e6 (0xd07c bytes) 2024-09-17T22:57:29.321 INFO:teuthology.orchestra.run.smithi159.stdout:380 write 0x82c5b18 thru 0x82d0ce7 (0xb1d0 bytes) 2024-09-17T22:57:29.336 INFO:teuthology.orchestra.run.smithi159.stdout:381 trunc from 0xeb9d89d to 0x3051f36 2024-09-17T22:57:29.343 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:29.340+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:29.354 INFO:teuthology.orchestra.run.smithi159.stdout:383 punch from 0x2eb23e5 to 0x2eb3e9b, (0x1ab6 bytes) 2024-09-17T22:57:29.372 INFO:teuthology.orchestra.run.smithi159.stdout:384 write 0xd2f4df4 thru 0xd3048fc (0xfb09 bytes) 2024-09-17T22:57:29.375 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.376 INFO:teuthology.orchestra.run.smithi159.stdout:387 writesame 0x903e103 thru 0x9042790 data_size 0xb(0x468e bytes) 2024-09-17T22:57:29.379 INFO:teuthology.orchestra.run.smithi159.stdout:388 writesame 0x95bb8ad thru 0x95c21b7 data_size 0x1(0x690b bytes) 2024-09-17T22:57:29.387 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.387 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.394 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.394 INFO:teuthology.orchestra.run.smithi159.stdout:390 writesame 0x4d01866 thru 0x4d08202 data_size 0x1(0x699d bytes) 2024-09-17T22:57:29.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:29.409 INFO:teuthology.orchestra.run.smithi159.stdout:391 write 0xd03f730 thru 0xd040ebd (0x178e bytes) 2024-09-17T22:57:29.413 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.413 INFO:teuthology.orchestra.run.smithi159.stdout:393 writesame 0x40c45a2 thru 0x40cb3d7 data_size 0x1(0x6e36 bytes) 2024-09-17T22:57:29.426 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.426 INFO:teuthology.orchestra.run.smithi159.stdout:397 trunc from 0xd3048fd to 0xd64ec67 2024-09-17T22:57:29.434 INFO:teuthology.orchestra.run.smithi159.stdout:398 trunc from 0xd64ec67 to 0xbc113ef 2024-09-17T22:57:29.443 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:29.441+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:29.459 INFO:teuthology.orchestra.run.smithi159.stdout:399 trunc from 0xbc113ef to 0x1823c59 2024-09-17T22:57:29.480 INFO:teuthology.orchestra.run.smithi159.stdout:400 read 0xd301db thru 0xd3c366 (0xc18c bytes) 2024-09-17T22:57:29.480 INFO:teuthology.orchestra.run.smithi159.stdout:401 read 0x7bc4b4 thru 0x7c2593 (0x60e0 bytes) 2024-09-17T22:57:29.481 INFO:teuthology.orchestra.run.smithi159.stdout:402 read 0x378882 thru 0x37fd49 (0x74c8 bytes) 2024-09-17T22:57:29.498 INFO:teuthology.orchestra.run.smithi159.stdout:403 write 0x941feff thru 0x942f21f (0xf321 bytes) 2024-09-17T22:57:29.501 INFO:teuthology.orchestra.run.smithi159.stdout:404 punch from 0x51f3586 to 0x51f6a47, (0x34c1 bytes) 2024-09-17T22:57:29.501 INFO:teuthology.orchestra.run.smithi159.stdout:405 read 0x4585560 thru 0x4591298 (0xbd39 bytes) 2024-09-17T22:57:29.502 INFO:teuthology.orchestra.run.smithi159.stdout:406 punch from 0x459f0ea to 0x45ab421, (0xc337 bytes) 2024-09-17T22:57:29.502 INFO:teuthology.orchestra.run.smithi159.stdout:407 read 0xd51d3b thru 0xd56393 (0x4659 bytes) 2024-09-17T22:57:29.502 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.502 INFO:teuthology.orchestra.run.smithi159.stdout:410 trunc from 0x942f220 to 0x90e1df6 2024-09-17T22:57:29.520 INFO:teuthology.orchestra.run.smithi159.stdout:411 write 0xbab80cb thru 0xbabc991 (0x48c7 bytes) 2024-09-17T22:57:29.522 INFO:teuthology.orchestra.run.smithi159.stdout:412 punch from 0x3c30246 to 0x3c3d40c, (0xd1c6 bytes) 2024-09-17T22:57:29.522 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.523 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.524 INFO:teuthology.orchestra.run.smithi159.stdout:415 trunc from 0xbabc992 to 0xa05e7ac 2024-09-17T22:57:29.531 INFO:teuthology.orchestra.run.smithi159.stdout:416 read 0x6c07dbe thru 0x6c0e9e2 (0x6c25 bytes) 2024-09-17T22:57:29.537 INFO:teuthology.orchestra.run.smithi159.stdout:418 write 0xa84e6ee thru 0xa852308 (0x3c1b bytes) 2024-09-17T22:57:29.539 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.539 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.544 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:29.542+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:29.546 INFO:teuthology.orchestra.run.smithi159.stdout:420 writesame 0xbc2164d thru 0xbc21d85 data_size 0x1(0x739 bytes) 2024-09-17T22:57:29.546 INFO:teuthology.orchestra.run.smithi159.stdout:421 write 0xa757527 thru 0xa765b8e (0xe668 bytes) 2024-09-17T22:57:29.549 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:29.549 INFO:teuthology.orchestra.run.smithi159.stdout:422 write 0x5cf131f thru 0x5cf8507 (0x71e9 bytes) 2024-09-17T22:57:29.553 INFO:teuthology.orchestra.run.smithi159.stdout:423 read 0x52f213b thru 0x52fc6d5 (0xa59b bytes) 2024-09-17T22:57:29.553 INFO:teuthology.orchestra.run.smithi159.stdout:424 write 0x512d2c5 thru 0x512dd38 (0xa74 bytes) 2024-09-17T22:57:29.556 INFO:teuthology.orchestra.run.smithi159.stdout:425 read 0xb5605cf thru 0xb56c422 (0xbe54 bytes) 2024-09-17T22:57:29.556 INFO:teuthology.orchestra.run.smithi159.stdout:426 writesame 0x1c64ed9 thru 0x1c69628 data_size 0x38(0x4750 bytes) 2024-09-17T22:57:29.558 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.559 INFO:teuthology.orchestra.run.smithi159.stdout:428 trunc from 0xbc21d86 to 0xaf844ef 2024-09-17T22:57:29.567 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.567 INFO:teuthology.orchestra.run.smithi159.stdout:430 writesame 0x6c83f0c thru 0x6c8bfda data_size 0x5(0x80cf bytes) 2024-09-17T22:57:29.572 INFO:teuthology.orchestra.run.smithi159.stdout:431 punch from 0xcba881 to 0xcc418e, (0x990d bytes) 2024-09-17T22:57:29.572 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.574 INFO:teuthology.orchestra.run.smithi159.stdout:433 trunc from 0xaf844ef to 0x956198a 2024-09-17T22:57:29.593 INFO:teuthology.orchestra.run.smithi159.stdout:434 write 0xa8d98c4 thru 0xa8e85e0 (0xed1d bytes) 2024-09-17T22:57:29.597 INFO:teuthology.orchestra.run.smithi159.stdout:435 punch from 0x7795c5f to 0x779866b, (0x2a0c bytes) 2024-09-17T22:57:29.597 INFO:teuthology.orchestra.run.smithi159.stdout:436 read 0x601388b thru 0x6022e66 (0xf5dc bytes) 2024-09-17T22:57:29.597 INFO:teuthology.orchestra.run.smithi159.stdout:437 write 0x4abcbc5 thru 0x4ac7135 (0xa571 bytes) 2024-09-17T22:57:29.608 INFO:teuthology.orchestra.run.smithi159.stdout:439 writesame 0xe5538d1 thru 0xe5621ff data_size 0x1(0xe92f bytes) 2024-09-17T22:57:29.608 INFO:teuthology.orchestra.run.smithi159.stdout:440 read 0x787fe2d thru 0x788c097 (0xc26b bytes) 2024-09-17T22:57:29.608 INFO:teuthology.orchestra.run.smithi159.stdout:441 write 0xb7603db thru 0xb76958b (0x91b1 bytes) 2024-09-17T22:57:29.611 INFO:teuthology.orchestra.run.smithi159.stdout:442 read 0x17b943b thru 0x17ba577 (0x113d bytes) 2024-09-17T22:57:29.612 INFO:teuthology.orchestra.run.smithi159.stdout:443 write 0xdbef6e0 thru 0xdbf0603 (0xf24 bytes) 2024-09-17T22:57:29.615 INFO:teuthology.orchestra.run.smithi159.stdout:444 punch from 0x95c5321 to 0x95cb240, (0x5f1f bytes) 2024-09-17T22:57:29.615 INFO:teuthology.orchestra.run.smithi159.stdout:446 write 0x609d627 thru 0x60ac6db (0xf0b5 bytes) 2024-09-17T22:57:29.618 INFO:teuthology.orchestra.run.smithi159.stdout:447 writesame 0x31d7880 thru 0x31dfbc7 data_size 0x4(0x8348 bytes) 2024-09-17T22:57:29.625 INFO:teuthology.orchestra.run.smithi159.stdout:448 punch from 0x49b160f to 0x49b21e9, (0xbda bytes) 2024-09-17T22:57:29.625 INFO:teuthology.orchestra.run.smithi159.stdout:452 write 0xd4d4cfb thru 0xd4dbabd (0x6dc3 bytes) 2024-09-17T22:57:29.627 INFO:teuthology.orchestra.run.smithi159.stdout:453 writesame 0x56b1d7f thru 0x56c0609 data_size 0x1(0xe88b bytes) 2024-09-17T22:57:29.634 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:29.659 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.659 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.660 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:29.657+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:29.667 INFO:teuthology.orchestra.run.smithi159.stdout:454 trunc from 0xe562200 to 0x2fc463a 2024-09-17T22:57:29.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:29.699 INFO:teuthology.orchestra.run.smithi159.stdout:456 write 0xed0dc90 thru 0xed0e093 (0x404 bytes) 2024-09-17T22:57:29.701 INFO:teuthology.orchestra.run.smithi159.stdout:457 read 0x2f5eff3 thru 0x2f628ae (0x38bc bytes) 2024-09-17T22:57:29.702 INFO:teuthology.orchestra.run.smithi159.stdout:459 read 0x20d26ad thru 0x20d5b42 (0x3496 bytes) 2024-09-17T22:57:29.702 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:29.702 INFO:teuthology.orchestra.run.smithi159.stdout:461 writesame 0x6d2cc3 thru 0x6decd6 data_size 0x26(0xc014 bytes) 2024-09-17T22:57:29.704 INFO:teuthology.orchestra.run.smithi159.stdout:462 write 0x5a782f thru 0x5a99f6 (0x21c8 bytes) 2024-09-17T22:57:29.705 INFO:teuthology.orchestra.run.smithi159.stdout:466 read 0xbbb84b thru 0xbbd473 (0x1c29 bytes) 2024-09-17T22:57:29.705 INFO:teuthology.orchestra.run.smithi159.stdout:467 write 0x698742e thru 0x698c72a (0x52fd bytes) 2024-09-17T22:57:29.708 INFO:teuthology.orchestra.run.smithi159.stdout:468 punch from 0x171b66 to 0x17f50d, (0xd9a7 bytes) 2024-09-17T22:57:29.708 INFO:teuthology.orchestra.run.smithi159.stdout:469 clone 6 order 21 su 16384 sc 12 2024-09-17T22:57:29.745 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:29.743+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:29.765 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:29.766 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:29.783 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:29.783 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:29.800 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:29.800 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:29.806 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:29.807 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:29.816 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:29.817 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:29.818 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:29.833 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:29.834 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:29.847 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:29.845+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:29.851 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:29.851 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:29.868 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:29.868 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:29.884 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:29.885 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:29.922 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:29.922 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:29.922 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:29.922 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:29.922 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:29.931 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:29.947 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:29.945+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:29.990 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:57:30.047 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone5 from 0xed0e094 (overlap 0x1823c59) to 0x9a7e8a 2024-09-17T22:57:30.048 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:30.045+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1029 3.19 e28/22 3:9e020cf5:::rbd_data.118840f89003.000000000000008e:head v 28'4, mlcod=27'1)", 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0593849689999999, 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051679889999999996, 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.090 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026575+0000", 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026576+0000", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026582+0000", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6200000000000001e-06 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026592+0000", 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9040000000000004e-06 2024-09-17T22:57:30.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026599+0000", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026643+0000", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4719999999999999e-05 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026664+0000", 2024-09-17T22:57:30.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1058000000000001e-05 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031712+0000", 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0050476200000000001 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031743+0000", 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1031000000000002e-05 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1031 3.15 e28/22 3:aa926c06:::rbd_data.118840f89003.000000000000008c:head v 28'6, mlcod=26'4)", 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0590392790000003, 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048447150000000003, 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026921+0000", 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3300000000000001e-07 2024-09-17T22:57:30.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026927+0000", 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2120000000000002e-06 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026934+0000", 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.255e-06 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026939+0000", 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9740000000000001e-06 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026979+0000", 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0130999999999999e-05 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.026991+0000", 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031727+0000", 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047360659999999997 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.031765+0000", 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8515e-05 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1107 3.8 3:1296b2d3:::rbd_data.118840f89003.00000000000000a4:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.099 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0471382939999998, 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005694859, 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1107 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038822+0000", 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038826+0000", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3349999999999998e-06 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038833+0000", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4279999999999999e-06 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038839+0000", 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8080000000000001e-06 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039055+0000", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000216157 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039144+0000", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9210999999999997e-05 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044517+0000", 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053722930000000002 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.103 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 150 entries 28'8 (0'0) error 3:a8b1de66:::rbd_data.118840f89003.00000000000000a6:head by client.4488.0:1112 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.046227118, 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047563509999999998, 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039733+0000", 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039734+0000", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039743+0000", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5230000000000001e-06 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039762+0000", 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9324999999999999e-05 2024-09-17T22:57:30.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039775+0000", 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2564e-05 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039833+0000", 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7954000000000001e-05 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044489+0000", 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046562790000000001 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1122 3.15 e28/22 3:a9e9a37d:::rbd_data.118840f89003.00000000000000a0:head v 28'9, mlcod=28'6)", 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0431629630000003, 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00465505, 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.109 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042797+0000", 2024-09-17T22:57:30.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042798+0000", 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.82e-07 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042802+0000", 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3030000000000001e-06 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042814+0000", 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2289999999999999e-05 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042821+0000", 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4049999999999997e-06 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042868+0000", 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7605999999999998e-05 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042882+0000", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3464e-05 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047413+0000", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045310159999999997 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047452+0000", 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9384000000000001e-05 2024-09-17T22:57:30.113 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1132 3.8 3:1296b2d3:::rbd_data.118840f89003.00000000000000a4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0409843700000003, 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053778960000000001, 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.114 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1132 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044976+0000", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1900000000000001e-07 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044981+0000", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044992+0000", 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0679000000000001e-05 2024-09-17T22:57:30.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.044999+0000", 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045085+0000", 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6337999999999999e-05 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045142+0000", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7055000000000002e-05 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045268+0000", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00012598899999999999 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050158+0000", 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048892739999999999 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050285+0000", 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00012700799999999999 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050331+0000", 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6696999999999997e-05 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050354+0000", 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2096e-05 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1133 3.15 e28/22 3:a8b1de66:::rbd_data.118840f89003.00000000000000a6:head v 28'10, mlcod=28'7)", 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0403850950000004, 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045667720000000002, 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.120 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045575+0000", 2024-09-17T22:57:30.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045576+0000", 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045581+0000", 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-06 2024-09-17T22:57:30.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045590+0000", 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0990000000000005e-06 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045594+0000", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045638+0000", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3658000000000001e-05 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.045651+0000", 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.253e-05 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050102+0000", 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004451048 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050142+0000", 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9993000000000001e-05 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.125 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1136 3.7 3:e2c60e86:::rbd_data.118840f89003.00000000000000a9:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0385770440000002, 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0056381740000000001, 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1136 2024-09-17T22:57:30.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047383+0000", 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047384+0000", 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8599999999999997e-07 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047490+0000", 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106618 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047505+0000", 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4175999999999999e-05 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047516+0000", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1396999999999999e-05 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047584+0000", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7830000000000006e-05 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047731+0000", 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014677000000000001 2024-09-17T22:57:30.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.053021+0000", 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052904969999999999 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1145 3.b 3:d691d729:::rbd_data.118840f89003.00000000000000ad:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0381389309999998, 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053637859999999997, 2024-09-17T22:57:30.130 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1145 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047821+0000", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047822+0000", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6700000000000003e-07 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047825+0000", 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0110000000000001e-06 2024-09-17T22:57:30.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047833+0000", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5040000000000002e-06 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047839+0000", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7049999999999996e-06 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047892+0000", 2024-09-17T22:57:30.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3168999999999999e-05 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048034+0000", 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014156499999999999 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.053185+0000", 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051512650000000004 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 153 entries 28'11 (0'0) error 3:af9fb172:::rbd_data.118840f89003.00000000000000ab:head by client.4488.0:1140 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'8)", 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0376868090000002, 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046930879999999998, 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048273+0000", 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048274+0000", 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048281+0000", 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3529999999999997e-06 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048300+0000", 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8899e-05 2024-09-17T22:57:30.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048313+0000", 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3254999999999999e-05 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.048373+0000", 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9873000000000001e-05 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.052966+0000", 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045932689999999997 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1148 3.8 3:11bb6454:::rbd_data.118840f89003.00000000000000ae:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0356991359999999, 2024-09-17T22:57:30.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005917276, 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1148 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:30.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050261+0000", 2024-09-17T22:57:30.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050262+0000", 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6799999999999999e-07 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050265+0000", 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1030000000000002e-06 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050274+0000", 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5130000000000002e-06 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050281+0000", 2024-09-17T22:57:30.146 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6239999999999996e-06 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050397+0000", 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000116493 2024-09-17T22:57:30.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.050483+0000", 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5624000000000006e-05 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056178+0000", 2024-09-17T22:57:30.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0056951509999999999 2024-09-17T22:57:30.149 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.150 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.150 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.150 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1179 3.b 3:d7b125d7:::rbd_data.118840f89003.00000000000000b6:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.151 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:30.151 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0293471470000002, 2024-09-17T22:57:30.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051194329999999996, 2024-09-17T22:57:30.151 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.151 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1179 2024-09-17T22:57:30.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056613+0000", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056614+0000", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0399999999999995e-07 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056617+0000", 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6440000000000002e-06 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056626+0000", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3440000000000001e-06 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056632+0000", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056706+0000", 2024-09-17T22:57:30.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4538999999999999e-05 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056844+0000", 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013722699999999999 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061732+0000", 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048886759999999998 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1180 3.8 3:11bb6454:::rbd_data.118840f89003.00000000000000ae:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.029122621, 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049744419999999999, 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.157 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1180 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056837+0000", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056838+0000", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5300000000000002e-07 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056843+0000", 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9350000000000002e-06 2024-09-17T22:57:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056856+0000", 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.295e-05 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056873+0000", 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6410999999999999e-05 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.056932+0000", 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9506000000000003e-05 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.057001+0000", 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9144000000000001e-05 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.057191+0000", 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00018997299999999999 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061466+0000", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0042747660000000002 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061722+0000", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025618700000000001 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061774+0000", 2024-09-17T22:57:30.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2021999999999998e-05 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.061812+0000", 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7595000000000002e-05 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1207 3.19 e28/22 3:9df5c97b:::rbd_data.118840f89003.00000000000000b7:head v 28'9, mlcod=28'6)", 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:30.163 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0212914739999999, 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048842470000000004, 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.164 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064669+0000", 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4600000000000004e-07 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064675+0000", 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6690000000000003e-06 2024-09-17T22:57:30.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064686+0000", 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1518999999999999e-05 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064691+0000", 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6149999999999999e-06 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064734+0000", 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3343000000000002e-05 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.064750+0000", 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6013999999999999e-05 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069520+0000", 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047699270000000002 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069553+0000", 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2413999999999998e-05 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1222 3.b 3:d3891ddd:::rbd_data.118840f89003.00000000000000c2:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0161347369999998, 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.005307137, 2024-09-17T22:57:30.168 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1222 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069825+0000", 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069826+0000", 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8599999999999998e-07 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069830+0000", 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7730000000000001e-06 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069840+0000", 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0318e-05 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069845+0000", 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069882+0000", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7070999999999998e-05 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.069956+0000", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3693000000000001e-05 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075132+0000", 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0051764039999999999 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.172 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:1240 3.b 3:d3891ddd:::rbd_data.118840f89003.00000000000000c2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0103867959999997, 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048908099999999998, 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:30.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 1240 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075573+0000", 2024-09-17T22:57:30.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075574+0000", 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075579+0000", 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.955e-06 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075588+0000", 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075594+0000", 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9549999999999997e-06 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075637+0000", 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3377999999999997e-05 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075688+0000", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810999999999999e-05 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.075804+0000", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011627499999999999 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080141+0000", 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0043364249999999997 2024-09-17T22:57:30.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080358+0000", 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021778800000000001 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080422+0000", 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899000000000006e-05 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.178 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.080464+0000", 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1625000000000002e-05 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.c epoch 28/22 rep_tid 36 entries 28'6 (0'0) error 3:30186e42:::rbd_data.118840f89003.00000000000000cd:head by client.4488.0:1259 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'3)", 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.0023879859999996, 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047667910000000003, 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.179 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083572+0000", 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083573+0000", 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9699999999999996e-07 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083580+0000", 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0960000000000003e-06 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083595+0000", 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5591999999999999e-05 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083604+0000", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1519999999999998e-06 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.083672+0000", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7488000000000002e-05 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.088339+0000", 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046668660000000004 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.182 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.19 epoch 28/22 rep_tid 45 entries 28'12 (0'0) error 3:9f6cdf93:::rbd_data.118840f89003.00000000000000ce:head by client.4488.0:1266 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'10)", 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9997402790000001, 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0050898829999999999, 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.183 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086220+0000", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086221+0000", 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-07 2024-09-17T22:57:30.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086229+0000", 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0949999999999996e-06 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086248+0000", 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9440999999999999e-05 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086262+0000", 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3709999999999999e-05 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.086342+0000", 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9957999999999996e-05 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.091310+0000", 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049677719999999996 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1287 3.19 e28/22 3:9f6cdf93:::rbd_data.118840f89003.00000000000000ce:head v 28'13, mlcod=28'11)", 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9918511429999999, 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0055062899999999996, 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:30.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094109+0000", 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094110+0000", 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4799999999999997e-07 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094115+0000", 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1839999999999998e-06 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094127+0000", 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1839e-05 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094141+0000", 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4078e-05 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094190+0000", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8984999999999999e-05 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094208+0000", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8583000000000001e-05 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099526+0000", 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053173450000000002 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099615+0000", 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9528000000000001e-05 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.15 epoch 28/22 rep_tid 167 entries 28'15 (0'0) error 3:ac37f788:::rbd_data.118840f89003.00000000000000d5:head by client.4488.0:1293 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'13)", 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9909927569999999, 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046150039999999998, 2024-09-17T22:57:30.191 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094967+0000", 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094968+0000", 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.16e-07 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094976+0000", 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8639999999999996e-06 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:30.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.094996+0000", 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0458999999999998e-05 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.095021+0000", 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4258000000000001e-05 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.095084+0000", 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3025000000000001e-05 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:30.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.099582+0000", 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0044985820000000001 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:30.195 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:30.197 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:30.197 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T22:57:30.197 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:30.194+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:30.297 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:30.295+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:30.301 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:30.301 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:30.301 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:30.301 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:30.310 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:30.348 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.348 INFO:teuthology.orchestra.run.smithi159.stdout:472 read 0x833114 thru 0x83da62 (0xa94f bytes) 2024-09-17T22:57:30.349 INFO:teuthology.orchestra.run.smithi159.stdout:473 writesame 0xd8a3027 thru 0xd8ac778 data_size 0x7(0x9752 bytes) 2024-09-17T22:57:30.374 INFO:teuthology.orchestra.run.smithi159.stdout:474 read 0x1861d71 thru 0x18678c3 (0x5b53 bytes) 2024-09-17T22:57:30.375 INFO:teuthology.orchestra.run.smithi159.stdout:476 write 0x44568e8 thru 0x4463d31 (0xd44a bytes) 2024-09-17T22:57:30.398 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:30.396+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:30.399 INFO:teuthology.orchestra.run.smithi159.stdout:477 write 0x288a119 thru 0x288c975 (0x285d bytes) 2024-09-17T22:57:30.417 INFO:teuthology.orchestra.run.smithi159.stdout:478 write 0x21fe4e thru 0x221216 (0x13c9 bytes) 2024-09-17T22:57:30.466 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:30.467 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:30.467 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:30.467 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:30.467 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:30.471 INFO:teuthology.orchestra.run.smithi159.stdout:479 writesame 0xb9811bc thru 0xb989cb2 data_size 0x5(0x8af7 bytes) 2024-09-17T22:57:30.477 INFO:teuthology.orchestra.run.smithi159.stdout:480 write 0xeaa0698 thru 0xeaa8caa (0x8613 bytes) 2024-09-17T22:57:30.479 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:30.479 INFO:teuthology.orchestra.run.smithi159.stdout:481 writesame 0x2dbf311 thru 0x2dc13e0 data_size 0x10(0x20d0 bytes) 2024-09-17T22:57:30.495 INFO:teuthology.orchestra.run.smithi159.stdout:482 read 0x73b6643 thru 0x73be70e (0x80cc bytes) 2024-09-17T22:57:30.496 INFO:teuthology.orchestra.run.smithi159.stdout:483 punch from 0x97dbe61 to 0x97e8793, (0xc932 bytes) 2024-09-17T22:57:30.496 INFO:teuthology.orchestra.run.smithi159.stdout:484 read 0x9a92290 thru 0x9a990c3 (0x6e34 bytes) 2024-09-17T22:57:30.496 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.496 INFO:teuthology.orchestra.run.smithi159.stdout:486 write 0x9e1287d thru 0x9e1dfe6 (0xb76a bytes) 2024-09-17T22:57:30.501 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:30.496+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:30.502 INFO:teuthology.orchestra.run.smithi159.stdout:488 read 0xe859b37 thru 0xe862abe (0x8f88 bytes) 2024-09-17T22:57:30.502 INFO:teuthology.orchestra.run.smithi159.stdout:490 write 0xff8fbe thru 0x100530b (0xc34e bytes) 2024-09-17T22:57:30.556 INFO:teuthology.orchestra.run.smithi159.stdout:491 punch from 0x7c93397 to 0x7c93bdf, (0x848 bytes) 2024-09-17T22:57:30.556 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.570 INFO:teuthology.orchestra.run.smithi159.stdout:493 trunc from 0xed0e094 to 0x20e7a74 2024-09-17T22:57:30.600 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:30.597+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1044 3.9 3:954c0790:::rbd_data.118840f89003.0000000000000090:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.574086232, 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045157670000000004, 2024-09-17T22:57:30.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1044 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027770+0000", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027771+0000", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.08e-06 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027776+0000", 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5260000000000001e-06 2024-09-17T22:57:30.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027791+0000", 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4222e-05 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027798+0000", 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9109999999999993e-06 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027854+0000", 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5788000000000002e-05 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.027941+0000", 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7026000000000003e-05 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.028149+0000", 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020735299999999999 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.029649+0000", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015007219999999999 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032187+0000", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025375430000000002 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032244+0000", 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6938000000000003e-05 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.032285+0000", 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1658e-05 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1102 3.1d e28/22 3:b8b32f6b:::rbd_data.118840f89003.000000000000009e:head v 28'7, mlcod=28'5)", 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5634520780000001, 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052004240000000004, 2024-09-17T22:57:30.611 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038404+0000", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3e-07 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038418+0000", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3531e-05 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038428+0000", 2024-09-17T22:57:30.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7990000000000005e-06 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038435+0000", 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7610000000000002e-06 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038481+0000", 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6449000000000002e-05 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038496+0000", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.485e-05 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043569+0000", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050729190000000004 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043604+0000", 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5485000000000003e-05 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.615 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.1 epoch 28/22 rep_tid 108 entries 28'3 (0'0) error 3:82735225:::rbd_data.118840f89003.00000000000000a3:head by client.4488.0:1104 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'1)", 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5629220339999996, 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049701559999999999, 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038934+0000", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038935+0000", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4000000000000001e-07 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038941+0000", 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5679999999999996e-06 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038956+0000", 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4800000000000001e-05 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.038965+0000", 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8629999999999997e-06 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039026+0000", 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1566e-05 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.043904+0000", 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048776189999999997 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1110 3.9 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.619 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5627537970000001, 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082675690000000007, 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1110 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039102+0000", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039103+0000", 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8800000000000001e-07 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039109+0000", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.711e-06 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039120+0000", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1082e-05 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039128+0000", 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7540000000000006e-06 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039168+0000", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0373000000000001e-05 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039262+0000", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3875999999999994e-05 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.047370+0000", 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081078850000000004 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1115 3.1a 3:581e07df:::rbd_data.118840f89003.000000000000009f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.562204296, 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050351440000000001, 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1115 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039652+0000", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.47e-07 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039657+0000", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7520000000000004e-06 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039669+0000", 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.134e-05 2024-09-17T22:57:30.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039676+0000", 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7400000000000004e-06 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039735+0000", 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9190000000000001e-05 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039818+0000", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2084999999999998e-05 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.039937+0000", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000119592 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044203+0000", 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0042659890000000004 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044621+0000", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041764299999999999 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044653+0000", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2239000000000001e-05 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.044687+0000", 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3726999999999997e-05 2024-09-17T22:57:30.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1152 3.3 3:c6fe237e:::rbd_data.118840f89003.00000000000000b0:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5514190430000001, 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0055420089999999997, 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1152 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050437+0000", 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050438+0000", 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2500000000000004e-07 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050442+0000", 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3470000000000001e-06 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050454+0000", 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1847e-05 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050462+0000", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050517+0000", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5124000000000003e-05 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050646+0000", 2024-09-17T22:57:30.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012879800000000001 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.055979+0000", 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053330620000000004 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.a epoch 28/22 rep_tid 26 entries 28'15 (0'0) error 3:5642bcd0:::rbd_data.118840f89003.00000000000000af:head by client.4488.0:1151 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'13)", 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5510605880000004, 2024-09-17T22:57:30.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045632449999999996, 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050795+0000", 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050796+0000", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050803+0000", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050820+0000", 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6798000000000001e-05 2024-09-17T22:57:30.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050834+0000", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4239e-05 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.050907+0000", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3351999999999998e-05 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.055359+0000", 2024-09-17T22:57:30.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044514699999999999 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.1c epoch 28/22 rep_tid 29 entries 28'10 (0'0) error 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head by client.4488.0:1154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'8)", 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5486968450000003, 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051115789999999998, 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.638 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053159+0000", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053160+0000", 2024-09-17T22:57:30.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053167+0000", 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6129999999999998e-06 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053193+0000", 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5592000000000002e-05 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053208+0000", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5213e-05 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053304+0000", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5874999999999999e-05 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058271+0000", 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049666909999999996 2024-09-17T22:57:30.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1159 3.3 3:c56ebdd4:::rbd_data.118840f89003.00000000000000b2:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5485881470000002, 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0050648550000000001, 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1159 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053268+0000", 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053269+0000", 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3300000000000001e-07 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053273+0000", 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053282+0000", 2024-09-17T22:57:30.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.764e-06 2024-09-17T22:57:30.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053299+0000", 2024-09-17T22:57:30.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7108999999999998e-05 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053348+0000", 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8436999999999997e-05 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.053465+0000", 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011670100000000001 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058333+0000", 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048680249999999998 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1177 3.3 3:c6fe237e:::rbd_data.118840f89003.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5453520469999997, 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051796150000000003, 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.650 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1177 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056504+0000", 2024-09-17T22:57:30.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056505+0000", 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0449999999999999e-06 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056510+0000", 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056522+0000", 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1457000000000001e-05 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056529+0000", 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056584+0000", 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5813999999999998e-05 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056667+0000", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2515999999999994e-05 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056824+0000", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000157331 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061278+0000", 2024-09-17T22:57:30.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044539879999999999 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061600+0000", 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032172 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061656+0000", 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5662999999999999e-05 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061683+0000", 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7895e-05 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1173 3.a e28/22 3:5642bcd0:::rbd_data.118840f89003.00000000000000af:head v 28'16, mlcod=28'14)", 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5451335290000001, 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045820289999999996, 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.656 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056722+0000", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056723+0000", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.68e-07 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056729+0000", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056739+0000", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0200000000000001e-05 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056746+0000", 2024-09-17T22:57:30.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2099999999999996e-06 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056792+0000", 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5729999999999998e-05 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.056807+0000", 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5017999999999999e-05 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061265+0000", 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044575509999999997 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061304+0000", 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9888999999999997e-05 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1186 3.3 3:c56ebdd4:::rbd_data.118840f89003.00000000000000b2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5429445900000003, 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053064840000000002, 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1186 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058911+0000", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058912+0000", 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3300000000000001e-07 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058916+0000", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1570000000000002e-06 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058926+0000", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058931+0000", 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0329999999999999e-06 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.058974+0000", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3426000000000001e-05 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059022+0000", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8344999999999999e-05 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059136+0000", 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113418 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063873+0000", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047369480000000004 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064085+0000", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021247099999999999 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064174+0000", 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9042999999999994e-05 2024-09-17T22:57:30.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064218+0000", 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3565e-05 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1187 3.1c e28/22 3:3f92f278:::rbd_data.118840f89003.00000000000000b1:head v 28'11, mlcod=28'9)", 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5424765650000003, 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045253089999999999, 2024-09-17T22:57:30.666 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059379+0000", 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059380+0000", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059385+0000", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.366e-06 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059397+0000", 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1739999999999999e-05 2024-09-17T22:57:30.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059404+0000", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.657e-06 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059440+0000", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6727999999999998e-05 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.059451+0000", 2024-09-17T22:57:30.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0458e-05 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063877+0000", 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044256410000000001 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.063905+0000", 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8073999999999999e-05 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.670 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1199 3.3 3:c7a091de:::rbd_data.118840f89003.00000000000000ba:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5398674039999998, 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049477330000000002, 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1199 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:30.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061989+0000", 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6899999999999996e-07 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.061995+0000", 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7200000000000003e-06 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062009+0000", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4287000000000001e-05 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062017+0000", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6029999999999999e-06 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062067+0000", 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9866e-05 2024-09-17T22:57:30.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.062172+0000", 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000105028 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066936+0000", 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0047642600000000002 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.675 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1217 3.3 3:c7a091de:::rbd_data.118840f89003.00000000000000ba:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5343356970000004, 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0056857569999999996, 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1217 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067520+0000", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067521+0000", 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067527+0000", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2709999999999998e-06 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067539+0000", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1516e-05 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067547+0000", 2024-09-17T22:57:30.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8550000000000002e-06 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067623+0000", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6631000000000005e-05 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067705+0000", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1144999999999996e-05 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067899+0000", 2024-09-17T22:57:30.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000194441 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.072399+0000", 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0044999630000000001 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073110+0000", 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00071130800000000001 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073166+0000", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5269999999999998e-05 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073206+0000", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0398999999999997e-05 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1223 3.3 3:c711af7a:::rbd_data.118840f89003.00000000000000c3:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:30.681 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5320019680000003, 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0052997110000000004, 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1223 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069854+0000", 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069855+0000", 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6000000000000003e-07 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069858+0000", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7170000000000002e-06 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069869+0000", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0356e-05 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069874+0000", 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.069918+0000", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4320000000000003e-05 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.070035+0000", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117278 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075154+0000", 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0051184769999999997 2024-09-17T22:57:30.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "pg_update_log_missing(3.a epoch 28/22 rep_tid 29 entries 28'17 (0'0) error 3:562220c4:::rbd_data.118840f89003.00000000000000c4:head by client.4488.0:1233 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'15)", 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5280742060000003, 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0049116289999999998, 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073782+0000", 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073789+0000", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073808+0000", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8649000000000001e-05 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073824+0000", 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5566999999999999e-05 2024-09-17T22:57:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073884+0000", 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0748999999999997e-05 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.078693+0000", 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0048088949999999997 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1241 3.3 3:c711af7a:::rbd_data.118840f89003.00000000000000c3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5261547870000003, 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.004755296, 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1241 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075701+0000", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075702+0000", 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9100000000000003e-07 2024-09-17T22:57:30.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075711+0000", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1390000000000001e-06 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075721+0000", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.029e-05 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075726+0000", 2024-09-17T22:57:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075769+0000", 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2805000000000001e-05 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075832+0000", 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.279e-05 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:30.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.075980+0000", 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014830099999999999 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079852+0000", 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003871846 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080356+0000", 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050418700000000002 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080413+0000", 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6645999999999999e-05 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080456+0000", 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3591999999999999e-05 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:1254 3.1c e28/22 3:3e41df91:::rbd_data.118840f89003.00000000000000cb:head v 28'13, mlcod=28'10)", 2024-09-17T22:57:30.695 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.5208761490000002, 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060906620000000002, 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080980+0000", 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3799999999999997e-07 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080993+0000", 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2381000000000001e-05 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081003+0000", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081009+0000", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4969999999999998e-06 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081063+0000", 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3591999999999998e-05 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.081085+0000", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2098000000000001e-05 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087031+0000", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059464030000000003 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.087070+0000", 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9214000000000001e-05 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:4703 3.f e32/22 3:f73e498f:::rbd_data.118846a2cbc0.0000000000000003:head v 32'12, mlcod=30'10)", 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.443531+0000", 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.15832447899999999, 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064766839999999999, 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.443531+0000", 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.443531+0000", 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.443533+0000", 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6029999999999999e-06 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.444455+0000", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00092212100000000003 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.444472+0000", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6898e-05 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.444480+0000", 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4429999999999997e-06 2024-09-17T22:57:30.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.444538+0000", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8674000000000001e-05 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.444557+0000", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8879e-05 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.449971+0000", 2024-09-17T22:57:30.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0054143339999999998 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.450008+0000", 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6732e-05 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:30.704 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:30.706 INFO:teuthology.orchestra.run.smithi159.stdout:495 read 0x11a3df0 thru 0x11b1703 (0xd914 bytes) 2024-09-17T22:57:30.706 INFO:teuthology.orchestra.run.smithi159.stdout:496 punch from 0x1a1d7a9 to 0x1a2334b, (0x5ba2 bytes) 2024-09-17T22:57:30.706 INFO:teuthology.orchestra.run.smithi159.stdout:497 read 0xad8260 thru 0xae5f00 (0xdca1 bytes) 2024-09-17T22:57:30.706 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.707 INFO:teuthology.orchestra.run.smithi159.stdout:499 write 0x264cc9b thru 0x2657902 (0xac68 bytes) 2024-09-17T22:57:30.707 INFO:teuthology.orchestra.run.smithi159.stdout:500 writesame 0x15eb769 thru 0x15f4af5 data_size 0x9(0x938d bytes) 2024-09-17T22:57:30.707 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:30.707 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:30.703+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:30.757 INFO:teuthology.orchestra.run.smithi159.stdout:501 write 0x57e44ad thru 0x57f214b (0xdc9f bytes) 2024-09-17T22:57:30.765 INFO:teuthology.orchestra.run.smithi159.stdout:502 read 0x2c07c33 thru 0x2c13291 (0xb65f bytes) 2024-09-17T22:57:30.782 INFO:teuthology.orchestra.run.smithi159.stdout:503 writesame 0xe37d769 thru 0xe3834fd data_size 0x1(0x5d95 bytes) 2024-09-17T22:57:30.782 INFO:teuthology.orchestra.run.smithi159.stdout:506 write 0x929290c thru 0x929cec3 (0xa5b8 bytes) 2024-09-17T22:57:30.790 INFO:teuthology.orchestra.run.smithi159.stdout:507 write 0x609b3fb thru 0x60a0274 (0x4e7a bytes) 2024-09-17T22:57:30.800 INFO:teuthology.orchestra.run.smithi159.stdout:509 write 0xc68cffd thru 0xc696c8d (0x9c91 bytes) 2024-09-17T22:57:30.806 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:30.804+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:30.808 INFO:teuthology.orchestra.run.smithi159.stdout:510 write 0x617196d thru 0x6175be6 (0x427a bytes) 2024-09-17T22:57:30.811 INFO:teuthology.orchestra.run.smithi159.stdout:511 read 0xe32a1a8 thru 0xe332a09 (0x8862 bytes) 2024-09-17T22:57:30.812 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.812 INFO:teuthology.orchestra.run.smithi159.stdout:514 read 0x5ddc482 thru 0x5de7046 (0xabc5 bytes) 2024-09-17T22:57:30.812 INFO:teuthology.orchestra.run.smithi159.stdout:515 write 0x6f23e7d thru 0x6f2bdc5 (0x7f49 bytes) 2024-09-17T22:57:30.815 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.815 INFO:teuthology.orchestra.run.smithi159.stdout:517 write 0x1aff819 thru 0x1b0c901 (0xd0e9 bytes) 2024-09-17T22:57:30.817 INFO:teuthology.orchestra.run.smithi159.stdout:518 read 0xb760550 thru 0xb765ca7 (0x5758 bytes) 2024-09-17T22:57:30.823 INFO:teuthology.orchestra.run.smithi159.stdout:519 write 0xeb3da61 thru 0xeb3f333 (0x18d3 bytes) 2024-09-17T22:57:30.823 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:30.823 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:30.824 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:30.824 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:30.826 INFO:teuthology.orchestra.run.smithi159.stdout:520 punch from 0xe40e4a1 to 0xe4112be, (0x2e1d bytes) 2024-09-17T22:57:30.831 INFO:teuthology.orchestra.run.smithi159.stdout:521 trunc from 0xeb3f334 to 0xa113108 2024-09-17T22:57:30.832 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:30.843 INFO:teuthology.orchestra.run.smithi159.stdout:523 punch from 0x921b61a to 0x9224fb5, (0x999b bytes) 2024-09-17T22:57:30.843 INFO:teuthology.orchestra.run.smithi159.stdout:524 writesame 0x18be083 thru 0x18ca071 data_size 0x1f(0xbfef bytes) 2024-09-17T22:57:30.847 INFO:teuthology.orchestra.run.smithi159.stdout:525 read 0x2022101 thru 0x202295d (0x85d bytes) 2024-09-17T22:57:30.853 INFO:teuthology.orchestra.run.smithi159.stdout:526 trunc from 0xa113108 to 0xd85d92b 2024-09-17T22:57:30.857 INFO:teuthology.orchestra.run.smithi159.stdout:527 write 0xc74ba36 thru 0xc755ce7 (0xa2b2 bytes) 2024-09-17T22:57:30.863 INFO:teuthology.orchestra.run.smithi159.stdout:528 write 0x8fc5673 thru 0x8fcb8f1 (0x627f bytes) 2024-09-17T22:57:30.872 INFO:teuthology.orchestra.run.smithi159.stdout:529 trunc from 0xd85d92b to 0x8be97b6 2024-09-17T22:57:30.884 INFO:teuthology.orchestra.run.smithi159.stdout:531 read 0xe687fe thru 0xe6fdc2 (0x75c5 bytes) 2024-09-17T22:57:30.884 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.896 INFO:teuthology.orchestra.run.smithi159.stdout:534 writesame 0xcef4366 thru 0xcefd6db data_size 0x32(0x9376 bytes) 2024-09-17T22:57:30.896 INFO:teuthology.orchestra.run.smithi159.stdout:535 read 0x317e926 thru 0x3183db5 (0x5490 bytes) 2024-09-17T22:57:30.896 INFO:teuthology.orchestra.run.smithi159.stdout:536 punch from 0x84451fc to 0x844dc26, (0x8a2a bytes) 2024-09-17T22:57:30.896 INFO:teuthology.orchestra.run.smithi159.stdout:537 writesame 0x696c58d thru 0x696f09a data_size 0xb(0x2b0e bytes) 2024-09-17T22:57:30.905 INFO:teuthology.orchestra.run.smithi159.stdout:538 writesame 0xe292132 thru 0xe293493 data_size 0x3(0x1362 bytes) 2024-09-17T22:57:30.905 INFO:teuthology.orchestra.run.smithi159.stdout:539 write 0x3902cc0 thru 0x39065ae (0x38ef bytes) 2024-09-17T22:57:30.907 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:30.905+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:30.911 INFO:teuthology.orchestra.run.smithi159.stdout:540 punch from 0x66d766a to 0x66e3825, (0xc1bb bytes) 2024-09-17T22:57:30.911 INFO:teuthology.orchestra.run.smithi159.stdout:541 read 0x5c7fe5f thru 0x5c86610 (0x67b2 bytes) 2024-09-17T22:57:30.911 INFO:teuthology.orchestra.run.smithi159.stdout:542 writesame 0x27d7656 thru 0x27d8688 data_size 0x1(0x1033 bytes) 2024-09-17T22:57:30.912 INFO:teuthology.orchestra.run.smithi159.stdout:543 read 0x5f4601a thru 0x5f4d6f6 (0x76dd bytes) 2024-09-17T22:57:30.913 INFO:teuthology.orchestra.run.smithi159.stdout:544 writesame 0xa8f284b thru 0xa8f7cd3 data_size 0x13(0x5489 bytes) 2024-09-17T22:57:30.917 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.917 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:30.917 INFO:teuthology.orchestra.run.smithi159.stdout:547 write 0x7ffc3c6 thru 0x8003dc6 (0x7a01 bytes) 2024-09-17T22:57:30.920 INFO:teuthology.orchestra.run.smithi159.stdout:548 read 0x21dc3c2 thru 0x21dd239 (0xe78 bytes) 2024-09-17T22:57:30.922 INFO:teuthology.orchestra.run.smithi159.stdout:549 trunc from 0xe293494 to 0xd1680b9 2024-09-17T22:57:30.927 INFO:teuthology.orchestra.run.smithi159.stdout:551 read 0xc38c126 thru 0xc398a7d (0xc958 bytes) 2024-09-17T22:57:30.927 INFO:teuthology.orchestra.run.smithi159.stdout:552 read 0x67376ab thru 0x6746aaa (0xf400 bytes) 2024-09-17T22:57:30.928 INFO:teuthology.orchestra.run.smithi159.stdout:553 punch from 0x1770f35 to 0x1773a68, (0x2b33 bytes) 2024-09-17T22:57:30.928 INFO:teuthology.orchestra.run.smithi159.stdout:556 read 0x9be110d thru 0x9be26a6 (0x159a bytes) 2024-09-17T22:57:30.928 INFO:teuthology.orchestra.run.smithi159.stdout:557 punch from 0x8134812 to 0x813f07d, (0xa86b bytes) 2024-09-17T22:57:30.928 INFO:teuthology.orchestra.run.smithi159.stdout:558 read 0xaaff089 thru 0xab093a7 (0xa31f bytes) 2024-09-17T22:57:30.929 INFO:teuthology.orchestra.run.smithi159.stdout:559 punch from 0x1807342 to 0x180d6a5, (0x6363 bytes) 2024-09-17T22:57:30.929 INFO:teuthology.orchestra.run.smithi159.stdout:560 read 0xa304548 thru 0xa3120d2 (0xdb8b bytes) 2024-09-17T22:57:30.929 INFO:teuthology.orchestra.run.smithi159.stdout:561 write 0x4ba700e thru 0x4baf980 (0x8973 bytes) 2024-09-17T22:57:30.933 INFO:teuthology.orchestra.run.smithi159.stdout:563 read 0x4d4bcbb thru 0x4d587ac (0xcaf2 bytes) 2024-09-17T22:57:30.933 INFO:teuthology.orchestra.run.smithi159.stdout:565 writesame 0x19f7ba5 thru 0x1a00440 data_size 0xe(0x889c bytes) 2024-09-17T22:57:30.937 INFO:teuthology.orchestra.run.smithi159.stdout:568 writesame 0x4404153 thru 0x440d2cd data_size 0x1(0x917b bytes) 2024-09-17T22:57:30.944 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:30.944 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:30.951 INFO:teuthology.orchestra.run.smithi159.stdout:569 clone 7 order 25 su 1048576 sc 7 2024-09-17T22:57:30.953 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:30.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:30.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:30.978 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T22:57:30.979 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:57:30.994 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:57:31.038 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:31.036+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:31.062 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone6 from 0xd1680b9 (overlap 0x20e7a74) to 0x34a5d8 2024-09-17T22:57:31.108 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:31.106+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.118 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1206 3.11 3:8eee0039:::rbd_data.118840f89003.00000000000000bc:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0505410929999996, 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90939317600000003, 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1206 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064328+0000", 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6899999999999997e-07 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064331+0000", 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0589999999999998e-06 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064338+0000", 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T22:57:31.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064342+0000", 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064385+0000", 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2216999999999999e-05 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:31.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064391+0000", 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.117e-06 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972129+0000", 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90773784000000002 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972245+0000", 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116396 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973721+0000", 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001475456 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1210 3.11 3:8af04179:::rbd_data.118840f89003.00000000000000bf:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0478886589999998, 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90700086899999999, 2024-09-17T22:57:31.124 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1210 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066984+0000", 2024-09-17T22:57:31.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7249999999999999e-06 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066996+0000", 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2337000000000001e-05 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067002+0000", 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6420000000000004e-06 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067055+0000", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2797000000000003e-05 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067064+0000", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0569999999999999e-06 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972638+0000", 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90557432699999996 2024-09-17T22:57:31.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972737+0000", 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8486999999999994e-05 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973981+0000", 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012439720000000001 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1235 3.11 3:8a065d9f:::rbd_data.118840f89003.00000000000000c6:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0413769610000001, 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90162610200000004, 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1235 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.130 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073492+0000", 2024-09-17T22:57:31.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073497+0000", 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8099999999999997e-06 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073510+0000", 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073517+0000", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0389999999999999e-06 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073571+0000", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4159000000000001e-05 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073580+0000", 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0880000000000001e-06 2024-09-17T22:57:31.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972930+0000", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89935040799999999 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973012+0000", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1601999999999996e-05 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975118+0000", 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021056640000000001 2024-09-17T22:57:31.134 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1249 3.11 3:89eed2ba:::rbd_data.118840f89003.00000000000000c8:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0349375509999996, 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89530808699999997, 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.135 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1249 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:31.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079932+0000", 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5000000000000002e-07 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079941+0000", 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7119999999999995e-06 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079953+0000", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1841000000000001e-05 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079961+0000", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5599999999999996e-06 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079999+0000", 2024-09-17T22:57:31.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8642e-05 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080005+0000", 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2029999999999997e-06 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973323+0000", 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89331756399999995 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973409+0000", 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5636999999999993e-05 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975239+0000", 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018302780000000001 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4069 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76138110599999997, 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012661786, 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4069 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353488+0000", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353490+0000", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8929999999999999e-06 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353495+0000", 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5280000000000003e-06 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353501+0000", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6509999999999998e-06 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366040+0000", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012539246 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366086+0000", 2024-09-17T22:57:31.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6369000000000003e-05 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366149+0000", 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2754000000000003e-05 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4070 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:31.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76136154, 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012774396, 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4070 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:31.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353509+0000", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.838e-06 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353515+0000", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353526+0000", 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1556000000000001e-05 2024-09-17T22:57:31.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366173+0000", 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012646749000000001 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366220+0000", 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7205e-05 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366281+0000", 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1177000000000001e-05 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4071 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2473984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76133575799999997, 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012885403, 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.150 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4071 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353535+0000", 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T22:57:31.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353542+0000", 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7479999999999996e-06 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353549+0000", 2024-09-17T22:57:31.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9609999999999999e-06 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366309+0000", 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012760102000000001 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366353+0000", 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4535e-05 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366418+0000", 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4627000000000003e-05 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4072 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76130816999999995, 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012992178, 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4072 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353561+0000", 2024-09-17T22:57:31.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5200000000000002e-07 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353563+0000", 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1629999999999999e-06 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353569+0000", 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1190000000000002e-06 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353574+0000", 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.729e-06 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366445+0000", 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012871356 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366492+0000", 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959000000000002e-05 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366552+0000", 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0399999999999998e-05 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4073 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76128742199999999, 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013104863, 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4073 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353583+0000", 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9180000000000001e-06 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353589+0000", 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5890000000000002e-06 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353595+0000", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.319e-06 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366580+0000", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012984961 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366622+0000", 2024-09-17T22:57:31.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2245e-05 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366686+0000", 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3485000000000006e-05 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4074 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76126596999999996, 2024-09-17T22:57:31.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013221811, 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4074 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353603+0000", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353605+0000", 2024-09-17T22:57:31.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9659999999999999e-06 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353611+0000", 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0100000000000001e-06 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353615+0000", 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4449999999999998e-06 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366716+0000", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013101218 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366764+0000", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7447000000000002e-05 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366824+0000", 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0392e-05 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.170 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4075 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76124713300000002, 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013339781, 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.171 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4075 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353622+0000", 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353624+0000", 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9700000000000002e-06 2024-09-17T22:57:31.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353629+0000", 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.977e-06 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353631+0000", 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366852+0000", 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013220849 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366898+0000", 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5534999999999997e-05 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366961+0000", 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3195999999999996e-05 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.175 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4076 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.761226709, 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013453582, 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4076 2024-09-17T22:57:31.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353644+0000", 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.72e-06 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353650+0000", 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.02e-06 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353653+0000", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0769999999999999e-06 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366988+0000", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013335157 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367036+0000", 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7642999999999998e-05 2024-09-17T22:57:31.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367095+0000", 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9684e-05 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4077 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76120576200000001, 2024-09-17T22:57:31.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013566762, 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4077 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353669+0000", 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5350000000000001e-06 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353674+0000", 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.605e-06 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353682+0000", 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6650000000000003e-06 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367122+0000", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013440275 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367173+0000", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1119999999999998e-05 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367229+0000", 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6267999999999997e-05 2024-09-17T22:57:31.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4078 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3850240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76118261499999995, 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013684187, 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4078 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1100000000000002e-07 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353694+0000", 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9950000000000005e-06 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353700+0000", 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1879999999999999e-06 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353703+0000", 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1779999999999999e-06 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367258+0000", 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013554237 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367305+0000", 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7630000000000003e-05 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367370+0000", 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4647999999999995e-05 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4079 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76115441100000003, 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013897985, 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4079 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8200000000000001e-07 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353716+0000", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339999999999999e-06 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353720+0000", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.721e-06 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353728+0000", 2024-09-17T22:57:31.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8590000000000005e-06 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367406+0000", 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013678832 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367474+0000", 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7979999999999996e-05 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367612+0000", 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000137677 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4080 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4374528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76113715100000001, 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014035901999999999, 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4080 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353733+0000", 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3200000000000001e-06 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353736+0000", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.326e-06 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353740+0000", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.653e-06 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367646+0000", 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013906452 2024-09-17T22:57:31.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367696+0000", 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0105000000000002e-05 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367767+0000", 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0834000000000007e-05 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4081 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76112564199999999, 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014163906, 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4081 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:31.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353744+0000", 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.158e-06 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353747+0000", 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9840000000000001e-06 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353750+0000", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.486e-06 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367797+0000", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014047389 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367848+0000", 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1459999999999999e-05 2024-09-17T22:57:31.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367907+0000", 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8220000000000002e-05 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4082 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4767744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76111452099999999, 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014293589000000001, 2024-09-17T22:57:31.203 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4082 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353755+0000", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0389999999999999e-06 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353759+0000", 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249999999999997e-06 2024-09-17T22:57:31.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353762+0000", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5040000000000001e-06 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367934+0000", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014172842 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367983+0000", 2024-09-17T22:57:31.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8599e-05 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368047+0000", 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4448000000000004e-05 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4083 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4964352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:31.208 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76110308400000004, 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014410081, 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4083 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:31.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353766+0000", 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353767+0000", 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353771+0000", 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9299999999999996e-06 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353773+0000", 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3580000000000001e-06 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368075+0000", 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014302265999999999 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368121+0000", 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5667000000000002e-05 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368175+0000", 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4558999999999997e-05 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4084 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 5160960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.76109238000000001, 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014534003, 2024-09-17T22:57:31.213 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4084 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353781+0000", 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1279999999999999e-06 2024-09-17T22:57:31.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353785+0000", 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389999999999999e-06 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353787+0000", 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.402e-06 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368196+0000", 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014409065 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368250+0000", 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368310+0000", 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9962e-05 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.217 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.218 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:31.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:31.219 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:31.217+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:31.244 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T22:57:31.274 INFO:teuthology.orchestra.run.smithi159.stdout:570 trunc from 0xd1680b9 to 0x52620e 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout: "epoch": 33, 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout: "fsid": "33081d94-d15e-4bd4-ba4c-30022eb1be7a", 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout: "created": "2024-09-17T22:56:50.056303+0000", 2024-09-17T22:57:31.307 INFO:teuthology.orchestra.run.smithi159.stdout: "modified": "2024-09-17T22:57:30.983458+0000", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "last_up_change": "2024-09-17T22:57:02.843283+0000", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "last_in_change": "2024-09-17T22:56:53.064036+0000", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_num": 5801984, 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_set": [ 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "noscrub", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pglog_hardlimit", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "purged_snapdirs", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_deletes", 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "sortbitwise" 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_version": 5, 2024-09-17T22:57:31.308 INFO:teuthology.orchestra.run.smithi159.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_snapshot": "", 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_max": 3, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd": 8, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "min_compat_client": "luminous", 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "require_osd_release": "squid", 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_crimson": false, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "pools": [ 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 1, 2024-09-17T22:57:31.309 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": ".mgr", 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:02.221581+0000", 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool", 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 1, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 1, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 1, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 1, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 1, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:31.311 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "17", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 0, 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 0, 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:31.312 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:31.313 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:31.314 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "options": { 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_max": 32, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_min": 1 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": {} 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 8, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 8, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.25, 2024-09-17T22:57:31.315 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 2, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "rbd", 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:08.859324+0000", 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:57:31.316 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 8, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 8, 2024-09-17T22:57:31.317 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 8, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 8, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 8, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "21", 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:31.318 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 2, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 21, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:31.320 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:31.321 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.9900000095367432, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.9900000095367432, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.75, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.75, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "pool_client.0", 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:17.992936+0000", 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:57:31.323 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 32, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 32, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 32, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 32, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 32, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:57:31.324 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "33", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 9, 2024-09-17T22:57:31.325 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 33, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:57:31.326 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:57:31.327 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.5, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.5, 2024-09-17T22:57:31.328 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 1, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.5, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.5, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 1, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "892321c3-409f-40e6-bfe1-c87f464d030f", 2024-09-17T22:57:31.329 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.330 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6808", 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6809", 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6810", 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6811", 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.332 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6814", 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6815", 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.333 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6812", 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6813", 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.334 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6809/2087020665", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6811/2087020665", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6815/2087020665", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6813/2087020665", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "cd777f15-a6ad-4f0c-820b-ef529dd9f008", 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.335 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.336 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6800", 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6801", 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.337 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6802", 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6803", 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6806", 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6807", 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6804", 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6805", 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.340 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6801/237398697", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6803/237398697", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6807/237398697", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6805/237398697", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "bd68e905-6e66-4735-9de8-66a99530420f", 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.341 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 18, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.342 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6816", 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6817", 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.343 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6818", 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6819", 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6822", 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6823", 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.345 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6820", 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6821", 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 927278608 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6817/927278608", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6819/927278608", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6823/927278608", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6821/927278608", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "28b3b7ca-d7d7-4ed9-8dfe-113d3579209c", 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.347 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.348 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6824", 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6825", 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.349 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6826", 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6827", 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6830", 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6831", 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6828", 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6829", 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6825/2987856795", 2024-09-17T22:57:31.352 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6827/2987856795", 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6831/2987856795", 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6829/2987856795", 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "d379c252-e66a-4d0e-af0a-81f6bda48c08", 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.354 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6826", 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6827", 2024-09-17T22:57:31.355 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6828", 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.356 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6829", 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6832", 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.357 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6833", 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6830", 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.358 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6831", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6827/2123531187", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6829/2123531187", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6833/2123531187", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6831/2123531187", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "e78a100f-1ff3-4af6-abd8-a069030ec951", 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 28, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.360 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6818", 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6819", 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.361 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6820", 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6821", 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.362 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6824", 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.363 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6825", 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6822", 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6823", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3064143709 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6819/3064143709", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6821/3064143709", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6825/3064143709", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6823/3064143709", 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.365 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "fc1af848-dfaa-4e40-a0af-8982c70f7ea9", 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:57:31.366 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6810", 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6811", 2024-09-17T22:57:31.367 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6812", 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.368 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6813", 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6816", 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6817", 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6814", 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6815", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6811/1505991260", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6813/1505991260", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6817/1505991260", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6815/1505991260", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.371 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "ccd75a18-ded4-4f75-a09a-926da5362a7f", 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 13, 2024-09-17T22:57:31.372 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 22, 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6800", 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6801", 2024-09-17T22:57:31.373 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6802", 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6803", 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6806", 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.375 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6807", 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6804", 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6805", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6801/585405482", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6803/585405482", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6807/585405482", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6805/585405482", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:57:31.377 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_xinfo": [ 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.101775+0000", 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.929708+0000", 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:57:31.379 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.562720+0000", 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.380 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.569375+0000", 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:57:00.002784+0000", 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.775667+0000", 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.382 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.558418+0000", 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:57:31.383 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.840790+0000", 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap": [ 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "pgid": "3.11", 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: 5, 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: 2 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.384 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_items": [], 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_primaries": [], 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_temp": [], 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_temp": [], 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "blocklist": {}, 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "range_blocklist": {}, 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profiles": { 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "default": { 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-failure-domain": "osd", 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "k": "2", 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "m": "1", 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin": "jerasure", 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: "technique": "reed_sol_van" 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps_queue": [ 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 2, 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 1 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "new_removed_snaps": [], 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "new_purged_snaps": [], 2024-09-17T22:57:31.386 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_node_flags": {}, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "device_class_flags": {}, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode": { 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_enabled": false, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_bucket_count": 0, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_bucket": 0 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:31.387 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:31.389 INFO:tasks.thrashosds.thrasher:j is {'epoch': 33, 'fsid': '33081d94-d15e-4bd4-ba4c-30022eb1be7a', 'created': '2024-09-17T22:56:50.056303+0000', 'modified': '2024-09-17T22:57:30.983458+0000', 'last_up_change': '2024-09-17T22:57:02.843283+0000', 'last_in_change': '2024-09-17T22:56:53.064036+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:57:02.221581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:57:08.859324+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:57:17.992936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '892321c3-409f-40e6-bfe1-c87f464d030f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6808', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6809', 'nonce': 2087020665}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6810', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6811', 'nonce': 2087020665}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6814', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6815', 'nonce': 2087020665}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6812', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6813', 'nonce': 2087020665}]}, 'public_addr': '172.21.15.159:6809/2087020665', 'cluster_addr': '172.21.15.159:6811/2087020665', 'heartbeat_back_addr': '172.21.15.159:6815/2087020665', 'heartbeat_front_addr': '172.21.15.159:6813/2087020665', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'cd777f15-a6ad-4f0c-820b-ef529dd9f008', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6800', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6801', 'nonce': 237398697}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6802', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6803', 'nonce': 237398697}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6806', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6807', 'nonce': 237398697}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6804', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6805', 'nonce': 237398697}]}, 'public_addr': '172.21.15.159:6801/237398697', 'cluster_addr': '172.21.15.159:6803/237398697', 'heartbeat_back_addr': '172.21.15.159:6807/237398697', 'heartbeat_front_addr': '172.21.15.159:6805/237398697', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bd68e905-6e66-4735-9de8-66a99530420f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6816', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6817', 'nonce': 927278608}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6818', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6819', 'nonce': 927278608}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6822', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6823', 'nonce': 927278608}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6820', 'nonce': 927278608}, {'type': 'v1', 'addr': '172.21.15.159:6821', 'nonce': 927278608}]}, 'public_addr': '172.21.15.159:6817/927278608', 'cluster_addr': '172.21.15.159:6819/927278608', 'heartbeat_back_addr': '172.21.15.159:6823/927278608', 'heartbeat_front_addr': '172.21.15.159:6821/927278608', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '28b3b7ca-d7d7-4ed9-8dfe-113d3579209c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6824', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6825', 'nonce': 2987856795}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6826', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6827', 'nonce': 2987856795}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6830', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6831', 'nonce': 2987856795}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6828', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6829', 'nonce': 2987856795}]}, 'public_addr': '172.21.15.159:6825/2987856795', 'cluster_addr': '172.21.15.159:6827/2987856795', 'heartbeat_back_addr': '172.21.15.159:6831/2987856795', 'heartbeat_front_addr': '172.21.15.159:6829/2987856795', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd379c252-e66a-4d0e-af0a-81f6bda48c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2123531187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2123531187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2123531187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2123531187}]}, 'public_addr': '172.21.15.171:6827/2123531187', 'cluster_addr': '172.21.15.171:6829/2123531187', 'heartbeat_back_addr': '172.21.15.171:6833/2123531187', 'heartbeat_front_addr': '172.21.15.171:6831/2123531187', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e78a100f-1ff3-4af6-abd8-a069030ec951', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 3064143709}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 3064143709}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 3064143709}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 3064143709}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 3064143709}]}, 'public_addr': '172.21.15.171:6819/3064143709', 'cluster_addr': '172.21.15.171:6821/3064143709', 'heartbeat_back_addr': '172.21.15.171:6825/3064143709', 'heartbeat_front_addr': '172.21.15.171:6823/3064143709', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'fc1af848-dfaa-4e40-a0af-8982c70f7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 1505991260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 1505991260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1505991260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 1505991260}]}, 'public_addr': '172.21.15.171:6811/1505991260', 'cluster_addr': '172.21.15.171:6813/1505991260', 'heartbeat_back_addr': '172.21.15.171:6817/1505991260', 'heartbeat_front_addr': '172.21.15.171:6815/1505991260', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ccd75a18-ded4-4f75-a09a-926da5362a7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 585405482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 585405482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 585405482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 585405482}]}, 'public_addr': '172.21.15.171:6801/585405482', 'cluster_addr': '172.21.15.171:6803/585405482', 'heartbeat_back_addr': '172.21.15.171:6807/585405482', 'heartbeat_front_addr': '172.21.15.171:6805/585405482', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.101775+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.929708+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.562720+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.569375+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:57:00.002784+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.775667+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.558418+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.840790+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.11', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:57:31.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:571 trunc from 0x52620e to 0xcbedb7c 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:572 write 0x82aa055 thru 0x82aea8c (0x4a38 bytes) 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:573 read 0x3b22973 thru 0x3b29988 (0x7016 bytes) 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:575 write 0x112d802 thru 0x113a209 (0xca08 bytes) 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:576 writesame 0x2ceea7f thru 0x2cf5731 data_size 0x1(0x6cb3 bytes) 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:577 trunc from 0xcbedb7c to 0xb35d0d2 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.391 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.391 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:31.386+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:31.392 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:31.392 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:31.392 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:31.392 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:31.392 INFO:teuthology.orchestra.run.smithi159.stdout:581 write 0xea22473 thru 0xea2a728 (0x82b6 bytes) 2024-09-17T22:57:31.395 INFO:teuthology.orchestra.run.smithi159.stdout:582 writesame 0x85faed1 thru 0x8606434 data_size 0x1a(0xb564 bytes) 2024-09-17T22:57:31.397 INFO:teuthology.orchestra.run.smithi159.stdout:583 read 0xb11f2e9 thru 0xb12dfc4 (0xecdc bytes) 2024-09-17T22:57:31.397 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.398 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:31.412 INFO:teuthology.orchestra.run.smithi159.stdout:585 trunc from 0xea2a729 to 0x28169a2 2024-09-17T22:57:31.426 INFO:teuthology.orchestra.run.smithi159.stdout:586 punch from 0x27789b7 to 0x2780b35, (0x817e bytes) 2024-09-17T22:57:31.426 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.438 INFO:teuthology.orchestra.run.smithi159.stdout:588 writesame 0x896f616 thru 0x897576f data_size 0x22(0x615a bytes) 2024-09-17T22:57:31.440 INFO:teuthology.orchestra.run.smithi159.stdout:590 write 0x97152b thru 0x977444 (0x5f1a bytes) 2024-09-17T22:57:31.451 INFO:teuthology.orchestra.run.smithi159.stdout:591 writesame 0xc389023 thru 0xc390976 data_size 0xa(0x7954 bytes) 2024-09-17T22:57:31.465 INFO:teuthology.orchestra.run.smithi159.stdout:592 trunc from 0xc390977 to 0xeb4b4d 2024-09-17T22:57:31.474 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.475 INFO:teuthology.orchestra.run.smithi159.stdout:594 read 0x153b01 thru 0x157c94 (0x4194 bytes) 2024-09-17T22:57:31.475 INFO:teuthology.orchestra.run.smithi159.stdout:595 trunc from 0xeb4b4d to 0xdf8992 2024-09-17T22:57:31.487 INFO:teuthology.orchestra.run.smithi159.stdout:598 trunc from 0xdf8992 to 0x479fb1b 2024-09-17T22:57:31.489 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:31.487+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:31.494 INFO:teuthology.orchestra.run.smithi159.stdout:599 trunc from 0x479fb1b to 0x344cb3a 2024-09-17T22:57:31.503 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.503 INFO:teuthology.orchestra.run.smithi159.stdout:604 read 0x1367cb8 thru 0x1375457 (0xd7a0 bytes) 2024-09-17T22:57:31.503 INFO:teuthology.orchestra.run.smithi159.stdout:605 read 0x557426 thru 0x55bcce (0x48a9 bytes) 2024-09-17T22:57:31.507 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:31.508 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:31.517 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:31.518 INFO:teuthology.orchestra.run.smithi159.stdout:606 write 0xa3a7f25 thru 0xa3af12c (0x7208 bytes) 2024-09-17T22:57:31.519 INFO:teuthology.orchestra.run.smithi159.stdout:607 punch from 0x2ccd706 to 0x2ccf20c, (0x1b06 bytes) 2024-09-17T22:57:31.519 INFO:teuthology.orchestra.run.smithi159.stdout:608 read 0x135a09c thru 0x135ebdf (0x4b44 bytes) 2024-09-17T22:57:31.529 INFO:teuthology.orchestra.run.smithi159.stdout:612 write 0xdbe7cd7 thru 0xdbe9162 (0x148c bytes) 2024-09-17T22:57:31.530 INFO:teuthology.orchestra.run.smithi159.stdout:613 writesame 0x81ab7bf thru 0x81acdd0 data_size 0x19(0x1612 bytes) 2024-09-17T22:57:31.532 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.532 INFO:teuthology.orchestra.run.smithi159.stdout:617 writesame 0x364a28e thru 0x36512d9 data_size 0x2(0x704c bytes) 2024-09-17T22:57:31.539 INFO:teuthology.orchestra.run.smithi159.stdout:618 punch from 0x30a6616 to 0x30b4bff, (0xe5e9 bytes) 2024-09-17T22:57:31.539 INFO:teuthology.orchestra.run.smithi159.stdout:620 read 0x4a331cc thru 0x4a33ac6 (0x8fb bytes) 2024-09-17T22:57:31.539 INFO:teuthology.orchestra.run.smithi159.stdout:621 punch from 0x7579d72 to 0x757c261, (0x24ef bytes) 2024-09-17T22:57:31.540 INFO:teuthology.orchestra.run.smithi159.stdout:625 write 0x5b5ff24 thru 0x5b64be3 (0x4cc0 bytes) 2024-09-17T22:57:31.541 INFO:teuthology.orchestra.run.smithi159.stdout:627 read 0x36f4d35 thru 0x36f5029 (0x2f5 bytes) 2024-09-17T22:57:31.541 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:31.541 INFO:teuthology.orchestra.run.smithi159.stdout:630 punch from 0x8025057 to 0x8031a46, (0xc9ef bytes) 2024-09-17T22:57:31.543 INFO:teuthology.orchestra.run.smithi159.stdout:631 trunc from 0xdbe9163 to 0xc2f423c 2024-09-17T22:57:31.551 INFO:teuthology.orchestra.run.smithi159.stdout:632 writesame 0x73f5875 thru 0x73f8225 data_size 0x1(0x29b1 bytes) 2024-09-17T22:57:31.556 INFO:teuthology.orchestra.run.smithi159.stdout:634 punch from 0x2e519ea to 0x2e5de54, (0xc46a bytes) 2024-09-17T22:57:31.557 INFO:teuthology.orchestra.run.smithi159.stdout:635 writesame 0x537cb15 thru 0x53827cc data_size 0x2e(0x5cb8 bytes) 2024-09-17T22:57:31.560 INFO:teuthology.orchestra.run.smithi159.stdout:636 punch from 0x6fb8a4 to 0x704f60, (0x96bc bytes) 2024-09-17T22:57:31.560 INFO:teuthology.orchestra.run.smithi159.stdout:637 clone 8 order 18 su 8192 sc 6 2024-09-17T22:57:31.590 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:31.587+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:6429 3.10 3:09ba62cc:::rbd_data.118846a2cbc0.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1916928~2806] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.703433+0000", 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.95634972200000001, 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010665284000000001, 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 6429 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.703433+0000", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.703433+0000", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.703434+0000", 2024-09-17T22:57:31.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.401e-06 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.705421+0000", 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019867859999999999 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.705440+0000", 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8836999999999999e-05 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.705450+0000", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0611999999999999e-05 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.705513+0000", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3070000000000004e-05 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.705686+0000", 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017321700000000001 2024-09-17T22:57:31.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.706097+0000", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041091400000000002 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.712428+0000", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063306289999999999 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.713972+0000", 2024-09-17T22:57:31.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001544332 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.714049+0000", 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6239e-05 2024-09-17T22:57:31.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.714098+0000", 2024-09-17T22:57:31.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9246999999999998e-05 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:6639 3.a 3:540c166a:::rbd_data.118846a2cbc0.0000000000000021:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.938883+0000", 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.72089957000000005, 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094825670000000008, 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:31.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 6639 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938883+0000", 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938883+0000", 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938883+0000", 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6799999999999999e-07 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938888+0000", 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938899+0000", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1012999999999999e-05 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938905+0000", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.938953+0000", 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8232000000000002e-05 2024-09-17T22:57:31.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.939051+0000", 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7983e-05 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.940566+0000", 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001515317 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.948195+0000", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0076290180000000004 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.948247+0000", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1357000000000003e-05 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.948311+0000", 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4633000000000003e-05 2024-09-17T22:57:31.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.948365+0000", 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.41e-05 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7805 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1392640~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.287051+0000", 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.37273094600000001, 2024-09-17T22:57:31.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0084049850000000002, 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7805 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287051+0000", 2024-09-17T22:57:31.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287051+0000", 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287052+0000", 2024-09-17T22:57:31.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287055+0000", 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6720000000000002e-06 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287062+0000", 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8430000000000004e-06 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287068+0000", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6579999999999999e-06 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.294902+0000", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078335269999999999 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.294980+0000", 2024-09-17T22:57:31.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8410000000000003e-05 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.295456+0000", 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00047635800000000002 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7806 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1998848~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.287074+0000", 2024-09-17T22:57:31.684 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.37270806000000001, 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0090968960000000001, 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7806 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287074+0000", 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287074+0000", 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287075+0000", 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287076+0000", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.756e-06 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287082+0000", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0159999999999997e-06 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.287088+0000", 2024-09-17T22:57:31.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8000000000000004e-06 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.295485+0000", 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0083968250000000001 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.295563+0000", 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8108999999999994e-05 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.296171+0000", 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00060804000000000001 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7936 3.10 3:09ba62cc:::rbd_data.118846a2cbc0.0000000000000001:9 [sparse-read 884736~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.289565+0000", 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.37021731600000002, 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0087847810000000002, 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.689 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7936 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289565+0000", 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289565+0000", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289565+0000", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289569+0000", 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:57:31.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289576+0000", 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.289582+0000", 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8900000000000004e-06 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297740+0000", 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081582619999999995 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297822+0000", 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1402000000000004e-05 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.298350+0000", 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00052803500000000005 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7937 3.10 3:09ba62cc:::rbd_data.118846a2cbc0.0000000000000001:9 [sparse-read 1490944~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.290064+0000", 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.369718663, 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089931909999999993, 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7937 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290064+0000", 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290064+0000", 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290067+0000", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0979999999999998e-06 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290070+0000", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3809999999999999e-06 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290079+0000", 2024-09-17T22:57:31.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3999999999999998e-06 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290086+0000", 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3010000000000003e-06 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.298382+0000", 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082957580000000003 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.298465+0000", 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3751999999999994e-05 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299057+0000", 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00059150100000000003 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.698 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7962 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 262144~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.290094+0000", 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36968859900000001, 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094708220000000003, 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.699 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7962 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290094+0000", 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290094+0000", 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290094+0000", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6600000000000002e-07 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290096+0000", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.198e-06 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290104+0000", 2024-09-17T22:57:31.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6199999999999999e-06 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290109+0000", 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9540000000000003e-06 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299090+0000", 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089812020000000006 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299187+0000", 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7162000000000002e-05 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299564+0000", 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00037722 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7963 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 868352~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.290116+0000", 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36966590500000002, 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099936769999999994, 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.704 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7963 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290116+0000", 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290116+0000", 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290117+0000", 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.46e-07 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290119+0000", 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:31.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290126+0000", 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0099999999999998e-06 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290131+0000", 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8119999999999998e-06 2024-09-17T22:57:31.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299614+0000", 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094830399999999999 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299695+0000", 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1624000000000003e-05 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300110+0000", 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041464099999999999 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7964 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1474560~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.290138+0000", 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36964460500000001, 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010580541000000001, 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.709 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7964 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290138+0000", 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290138+0000", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290138+0000", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290141+0000", 2024-09-17T22:57:31.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3400000000000002e-06 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290148+0000", 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6930000000000001e-06 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290153+0000", 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9930000000000003e-06 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300139+0000", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099857709999999992 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300220+0000", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0661999999999994e-05 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300718+0000", 2024-09-17T22:57:31.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00049861999999999997 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8039 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.290726+0000", 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36905652, 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010188944, 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.714 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8039 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290726+0000", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290726+0000", 2024-09-17T22:57:31.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290726+0000", 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9500000000000003e-07 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290729+0000", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2809999999999998e-06 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290735+0000", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2940000000000002e-06 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.290756+0000", 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1038000000000001e-05 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300748+0000", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099926230000000008 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300828+0000", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9653000000000005e-05 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300915+0000", 2024-09-17T22:57:31.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6559999999999998e-05 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8040 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.291737+0000", 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36804572499999999, 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093678010000000003, 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8040 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291737+0000", 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291737+0000", 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291737+0000", 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9100000000000003e-07 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291744+0000", 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4919999999999998e-06 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291753+0000", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5599999999999999e-06 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291762+0000", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1459999999999996e-06 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300942+0000", 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0091802740000000004 2024-09-17T22:57:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301026+0000", 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3758999999999995e-05 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301104+0000", 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8778999999999999e-05 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8041 3.10 3.60dfabb0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.291777+0000", 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36800491400000002, 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093579909999999995, 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8041 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291777+0000", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291777+0000", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291778+0000", 2024-09-17T22:57:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9800000000000003e-07 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291783+0000", 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2820000000000004e-06 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291791+0000", 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3690000000000001e-06 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291796+0000", 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5949999999999998e-06 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301128+0000", 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093317269999999997 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301135+0000", 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8042 3.10 3.60dfabb0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.291804+0000", 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36797834000000001, 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093620079999999998, 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8042 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291804+0000", 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291804+0000", 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291804+0000", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291807+0000", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8490000000000002e-06 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291813+0000", 2024-09-17T22:57:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9739999999999999e-06 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.291818+0000", 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4819999999999996e-06 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301159+0000", 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093410420000000008 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.301166+0000", 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2010000000000002e-06 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8292 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 425984~98304] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.303816+0000", 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.35596644199999999, 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011350591, 2024-09-17T22:57:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8292 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303816+0000", 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303816+0000", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303816+0000", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303818+0000", 2024-09-17T22:57:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0990000000000001e-06 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303825+0000", 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1410000000000002e-06 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303829+0000", 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6949999999999999e-06 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.311877+0000", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0080473339999999997 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.311957+0000", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0764000000000001e-05 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.315166+0000", 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0032090700000000001 2024-09-17T22:57:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8293 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1048576~81920] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.35594714999999999, 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014439354, 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8293 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303836+0000", 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303837+0000", 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303849+0000", 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1089e-05 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303854+0000", 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9740000000000001e-06 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.315219+0000", 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011365831 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.315321+0000", 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101469 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318274+0000", 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029536359999999999 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8294 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1654784~81920] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.35592210099999999, 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016892707999999999, 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8294 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303861+0000", 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303867+0000", 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303878+0000", 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0560999999999999e-05 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303883+0000", 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318353+0000", 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470304 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318442+0000", 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9520999999999999e-05 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320753+0000", 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023103939999999999 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8367 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 262144~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.35542700500000002, 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016773656000000001, 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8367 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304356+0000", 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0500000000000003e-07 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304359+0000", 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304367+0000", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304381+0000", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4491000000000001e-05 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320796+0000", 2024-09-17T22:57:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016414173000000001 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320859+0000", 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3272e-05 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321129+0000", 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027004600000000002 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8368 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 868352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.354435057, 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015923571000000001, 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8368 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305348+0000", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9699999999999994e-07 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305353+0000", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000003e-06 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305362+0000", 2024-09-17T22:57:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9279999999999999e-06 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305371+0000", 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6090000000000001e-06 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321157+0000", 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015786517 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321214+0000", 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7021000000000002e-05 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321271+0000", 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6579000000000002e-05 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8369 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.354315561, 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01592536, 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8369 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1799999999999995e-07 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305471+0000", 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5889999999999999e-06 2024-09-17T22:57:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305478+0000", 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5519999999999999e-06 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305485+0000", 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1879999999999999e-06 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321295+0000", 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015810267999999999 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321339+0000", 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4184000000000003e-05 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321392+0000", 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2961000000000003e-05 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8430 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1048576~30408704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.312767+0000", 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.34701541800000002, 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011901297999999999, 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8430 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.312767+0000", 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.312767+0000", 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.312769+0000", 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.804e-06 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.313951+0000", 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011824209999999999 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.313962+0000", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1375e-05 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.313968+0000", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5629999999999999e-06 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321407+0000", 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074385959999999996 2024-09-17T22:57:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321504+0000", 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6941999999999994e-05 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321688+0000", 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018432299999999999 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.323017+0000", 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001329422 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.324595+0000", 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015776309999999999 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.324630+0000", 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4811999999999998e-05 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.324668+0000", 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8408999999999998e-05 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:31.768 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:31.770 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:57:31.771 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:57:31.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:31.773 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-17T22:57:30.993211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170255736,"num_objects":148,"num_object_clones":11,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":95,"num_read":3020,"num_read_kb":3018,"num_write":658,"num_write_kb":4856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":645,"ondisk_log_size":645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225780,"kb_used_data":5372,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748940,"statfs":{"total":773094113280,"available":772862914560,"internally_reserved":0,"allocated":5500928,"data_stored":3753284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":121469465,"num_objects":120,"num_object_clones":11,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":79,"num_read":2781,"num_read_kb":2872,"num_write":455,"num_write_kb":1982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026778"},"pg_stats":[{"pgid":"3.1f","version":"29'81","reported_seq":262,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.986838+0000","last_change":"2024-09-17T22:57:21.855741+0000","last_active":"2024-09-17T22:57:26.986838+0000","last_peered":"2024-09-17T22:57:26.986838+0000","last_clean":"2024-09-17T22:57:26.986838+0000","last_became_active":"2024-09-17T22:57:19.849385+0000","last_became_peered":"2024-09-17T22:57:19.849385+0000","last_unstale":"2024-09-17T22:57:26.986838+0000","last_undegraded":"2024-09-17T22:57:26.986838+0000","last_fullsized":"2024-09-17T22:57:26.986838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:07:53.938378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001658269,"stat_sum":{"num_bytes":1959502,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":658,"num_read_kb":650,"num_write":111,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"28'5","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.975826+0000","last_change":"2024-09-17T22:57:21.844862+0000","last_active":"2024-09-17T22:57:26.975826+0000","last_peered":"2024-09-17T22:57:26.975826+0000","last_clean":"2024-09-17T22:57:26.975826+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:57:26.975826+0000","last_undegraded":"2024-09-17T22:57:26.975826+0000","last_fullsized":"2024-09-17T22:57:26.975826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:05.945158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042388700000000002,"stat_sum":{"num_bytes":91,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":8,"num_read_kb":3,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"28'11","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.986355+0000","last_change":"2024-09-17T22:57:21.851512+0000","last_active":"2024-09-17T22:57:27.986355+0000","last_peered":"2024-09-17T22:57:27.986355+0000","last_clean":"2024-09-17T22:57:27.986355+0000","last_became_active":"2024-09-17T22:57:19.853868+0000","last_became_peered":"2024-09-17T22:57:19.853868+0000","last_unstale":"2024-09-17T22:57:27.986355+0000","last_undegraded":"2024-09-17T22:57:27.986355+0000","last_fullsized":"2024-09-17T22:57:27.986355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:27.217573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010535799999999999,"stat_sum":{"num_bytes":260891,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":6,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"30'15","reported_seq":67,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.105602+0000","last_change":"2024-09-17T22:57:21.851236+0000","last_active":"2024-09-17T22:57:28.105602+0000","last_peered":"2024-09-17T22:57:28.105602+0000","last_clean":"2024-09-17T22:57:28.105602+0000","last_became_active":"2024-09-17T22:57:19.852990+0000","last_became_peered":"2024-09-17T22:57:19.852990+0000","last_unstale":"2024-09-17T22:57:28.105602+0000","last_undegraded":"2024-09-17T22:57:28.105602+0000","last_fullsized":"2024-09-17T22:57:28.105602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:20:43.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089476699999999998,"stat_sum":{"num_bytes":546272,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":34,"num_read_kb":30,"num_write":15,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"28'9","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.967960+0000","last_change":"2024-09-17T22:57:21.855970+0000","last_active":"2024-09-17T22:57:26.967960+0000","last_peered":"2024-09-17T22:57:26.967960+0000","last_clean":"2024-09-17T22:57:26.967960+0000","last_became_active":"2024-09-17T22:57:19.852369+0000","last_became_peered":"2024-09-17T22:57:19.852369+0000","last_unstale":"2024-09-17T22:57:26.967960+0000","last_undegraded":"2024-09-17T22:57:26.967960+0000","last_fullsized":"2024-09-17T22:57:26.967960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:36.235139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134198,"stat_sum":{"num_bytes":3125597,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":34,"num_read_kb":116,"num_write":8,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"30'23","reported_seq":239,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.208447+0000","last_change":"2024-09-17T22:57:21.845702+0000","last_active":"2024-09-17T22:57:28.208447+0000","last_peered":"2024-09-17T22:57:28.208447+0000","last_clean":"2024-09-17T22:57:28.208447+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:57:28.208447+0000","last_undegraded":"2024-09-17T22:57:28.208447+0000","last_fullsized":"2024-09-17T22:57:28.208447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:20:41.701167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822500000000001,"stat_sum":{"num_bytes":9471734,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":231,"num_read_kb":393,"num_write":19,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"28'17","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.984624+0000","last_change":"2024-09-17T22:57:21.854734+0000","last_active":"2024-09-17T22:57:26.984624+0000","last_peered":"2024-09-17T22:57:26.984624+0000","last_clean":"2024-09-17T22:57:26.984624+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:57:26.984624+0000","last_undegraded":"2024-09-17T22:57:26.984624+0000","last_fullsized":"2024-09-17T22:57:26.984624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:27:44.894249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083302300000000001,"stat_sum":{"num_bytes":131072,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":9,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"26'35","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.975493+0000","last_change":"2024-09-17T22:57:21.845501+0000","last_active":"2024-09-17T22:57:26.975493+0000","last_peered":"2024-09-17T22:57:26.975493+0000","last_clean":"2024-09-17T22:57:26.975493+0000","last_became_active":"2024-09-17T22:57:19.848501+0000","last_became_peered":"2024-09-17T22:57:19.848501+0000","last_unstale":"2024-09-17T22:57:26.975493+0000","last_undegraded":"2024-09-17T22:57:26.975493+0000","last_fullsized":"2024-09-17T22:57:26.975493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:28.136920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090556399999999998,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":171,"num_read_kb":136,"num_write":58,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"28'5","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.968117+0000","last_change":"2024-09-17T22:57:21.855764+0000","last_active":"2024-09-17T22:57:26.968117+0000","last_peered":"2024-09-17T22:57:26.968117+0000","last_clean":"2024-09-17T22:57:26.968117+0000","last_became_active":"2024-09-17T22:57:19.849370+0000","last_became_peered":"2024-09-17T22:57:19.849370+0000","last_unstale":"2024-09-17T22:57:26.968117+0000","last_undegraded":"2024-09-17T22:57:26.968117+0000","last_fullsized":"2024-09-17T22:57:26.968117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:21.323410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001474541,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"30'15","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.212307+0000","last_change":"2024-09-17T22:57:21.851549+0000","last_active":"2024-09-17T22:57:28.212307+0000","last_peered":"2024-09-17T22:57:28.212307+0000","last_clean":"2024-09-17T22:57:28.212307+0000","last_became_active":"2024-09-17T22:57:19.850912+0000","last_became_peered":"2024-09-17T22:57:19.850912+0000","last_unstale":"2024-09-17T22:57:28.212307+0000","last_undegraded":"2024-09-17T22:57:28.212307+0000","last_fullsized":"2024-09-17T22:57:28.212307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:18:09.038757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092406,"stat_sum":{"num_bytes":20079223,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":232,"num_read_kb":57,"num_write":10,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"28'16","reported_seq":105,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.105486+0000","last_change":"2024-09-17T22:57:22.703365+0000","last_active":"2024-09-17T22:57:26.105486+0000","last_peered":"2024-09-17T22:57:26.105486+0000","last_clean":"2024-09-17T22:57:26.105486+0000","last_became_active":"2024-09-17T22:57:19.845355+0000","last_became_peered":"2024-09-17T22:57:19.845355+0000","last_unstale":"2024-09-17T22:57:26.105486+0000","last_undegraded":"2024-09-17T22:57:26.105486+0000","last_fullsized":"2024-09-17T22:57:26.105486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:12.869801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052696899999999996,"stat_sum":{"num_bytes":28264363,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":71,"num_read_kb":13,"num_write":11,"num_write_kb":186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"30'35","reported_seq":70,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.980736+0000","last_change":"2024-09-17T22:57:21.846023+0000","last_active":"2024-09-17T22:57:27.980736+0000","last_peered":"2024-09-17T22:57:27.980736+0000","last_clean":"2024-09-17T22:57:27.980736+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:57:27.980736+0000","last_undegraded":"2024-09-17T22:57:27.980736+0000","last_fullsized":"2024-09-17T22:57:27.980736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:26.402018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055329399999999996,"stat_sum":{"num_bytes":305863,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":129,"num_read_kb":96,"num_write":47,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.028192+0000","last_change":"2024-09-17T22:57:11.898422+0000","last_active":"2024-09-17T22:57:28.028192+0000","last_peered":"2024-09-17T22:57:28.028192+0000","last_clean":"2024-09-17T22:57:28.028192+0000","last_became_active":"2024-09-17T22:57:09.887327+0000","last_became_peered":"2024-09-17T22:57:09.887327+0000","last_unstale":"2024-09-17T22:57:28.028192+0000","last_undegraded":"2024-09-17T22:57:28.028192+0000","last_fullsized":"2024-09-17T22:57:28.028192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:50:22.559392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'16","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.021677+0000","last_change":"2024-09-17T22:57:21.854577+0000","last_active":"2024-09-17T22:57:27.021677+0000","last_peered":"2024-09-17T22:57:27.021677+0000","last_clean":"2024-09-17T22:57:27.021677+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:57:27.021677+0000","last_undegraded":"2024-09-17T22:57:27.021677+0000","last_fullsized":"2024-09-17T22:57:27.021677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:09.835505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053521000000000005,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":66,"num_read_kb":53,"num_write":11,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.984751+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:57:26.984751+0000","last_peered":"2024-09-17T22:57:26.984751+0000","last_clean":"2024-09-17T22:57:26.984751+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:57:26.984751+0000","last_undegraded":"2024-09-17T22:57:26.984751+0000","last_fullsized":"2024-09-17T22:57:26.984751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"29'26","reported_seq":184,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.027305+0000","last_change":"2024-09-17T22:57:21.845376+0000","last_active":"2024-09-17T22:57:27.027305+0000","last_peered":"2024-09-17T22:57:27.027305+0000","last_clean":"2024-09-17T22:57:27.027305+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:57:27.027305+0000","last_undegraded":"2024-09-17T22:57:27.027305+0000","last_fullsized":"2024-09-17T22:57:27.027305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:44.682569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000836865,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":386,"num_read_kb":327,"num_write":26,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:25.963996+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:57:25.963996+0000","last_peered":"2024-09-17T22:57:25.963996+0000","last_clean":"2024-09-17T22:57:25.963996+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:57:25.963996+0000","last_undegraded":"2024-09-17T22:57:25.963996+0000","last_fullsized":"2024-09-17T22:57:25.963996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"28'8","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.984662+0000","last_change":"2024-09-17T22:57:21.854702+0000","last_active":"2024-09-17T22:57:26.984662+0000","last_peered":"2024-09-17T22:57:26.984662+0000","last_clean":"2024-09-17T22:57:26.984662+0000","last_became_active":"2024-09-17T22:57:19.951388+0000","last_became_peered":"2024-09-17T22:57:19.951388+0000","last_unstale":"2024-09-17T22:57:26.984662+0000","last_undegraded":"2024-09-17T22:57:26.984662+0000","last_fullsized":"2024-09-17T22:57:26.984662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:33.728955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080280599999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3,"num_read_kb":0,"num_write":6,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.984700+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:57:26.984700+0000","last_peered":"2024-09-17T22:57:26.984700+0000","last_clean":"2024-09-17T22:57:26.984700+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:57:26.984700+0000","last_undegraded":"2024-09-17T22:57:26.984700+0000","last_fullsized":"2024-09-17T22:57:26.984700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'7","reported_seq":76,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.196579+0000","last_change":"2024-09-17T22:57:21.851751+0000","last_active":"2024-09-17T22:57:28.196579+0000","last_peered":"2024-09-17T22:57:28.196579+0000","last_clean":"2024-09-17T22:57:28.196579+0000","last_became_active":"2024-09-17T22:57:19.853015+0000","last_became_peered":"2024-09-17T22:57:19.853015+0000","last_unstale":"2024-09-17T22:57:28.196579+0000","last_undegraded":"2024-09-17T22:57:28.196579+0000","last_fullsized":"2024-09-17T22:57:28.196579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:37:30.474066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723509999999999,"stat_sum":{"num_bytes":5495233,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":43,"num_read_kb":55,"num_write":6,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"28'11","reported_seq":45,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.980676+0000","last_change":"2024-09-17T22:57:21.845990+0000","last_active":"2024-09-17T22:57:27.980676+0000","last_peered":"2024-09-17T22:57:27.980676+0000","last_clean":"2024-09-17T22:57:27.980676+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:57:27.980676+0000","last_undegraded":"2024-09-17T22:57:27.980676+0000","last_fullsized":"2024-09-17T22:57:27.980676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:48:45.101020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058788700000000004,"stat_sum":{"num_bytes":14487217,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6,"num_read_kb":19,"num_write":6,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.986185+0000","last_change":"2024-09-17T22:57:11.900419+0000","last_active":"2024-09-17T22:57:27.986185+0000","last_peered":"2024-09-17T22:57:27.986185+0000","last_clean":"2024-09-17T22:57:27.986185+0000","last_became_active":"2024-09-17T22:57:09.891032+0000","last_became_peered":"2024-09-17T22:57:09.891032+0000","last_unstale":"2024-09-17T22:57:27.986185+0000","last_undegraded":"2024-09-17T22:57:27.986185+0000","last_fullsized":"2024-09-17T22:57:27.986185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:07.358184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087551800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"29'32","reported_seq":248,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.044089+0000","last_change":"2024-09-17T22:57:21.854973+0000","last_active":"2024-09-17T22:57:27.044089+0000","last_peered":"2024-09-17T22:57:27.044089+0000","last_clean":"2024-09-17T22:57:27.044089+0000","last_became_active":"2024-09-17T22:57:19.951410+0000","last_became_peered":"2024-09-17T22:57:19.951410+0000","last_unstale":"2024-09-17T22:57:27.044089+0000","last_undegraded":"2024-09-17T22:57:27.044089+0000","last_fullsized":"2024-09-17T22:57:27.044089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:26.017698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841561,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":569,"num_read_kb":480,"num_write":28,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.967751+0000","last_change":"2024-09-17T22:57:12.700487+0000","last_active":"2024-09-17T22:57:26.967751+0000","last_peered":"2024-09-17T22:57:26.967751+0000","last_clean":"2024-09-17T22:57:26.967751+0000","last_became_active":"2024-09-17T22:57:09.890973+0000","last_became_peered":"2024-09-17T22:57:09.890973+0000","last_unstale":"2024-09-17T22:57:26.967751+0000","last_undegraded":"2024-09-17T22:57:26.967751+0000","last_fullsized":"2024-09-17T22:57:26.967751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:05.986370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030900600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"28'7","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.967950+0000","last_change":"2024-09-17T22:57:21.854980+0000","last_active":"2024-09-17T22:57:26.967950+0000","last_peered":"2024-09-17T22:57:26.967950+0000","last_clean":"2024-09-17T22:57:26.967950+0000","last_became_active":"2024-09-17T22:57:19.852417+0000","last_became_peered":"2024-09-17T22:57:19.852417+0000","last_unstale":"2024-09-17T22:57:26.967950+0000","last_undegraded":"2024-09-17T22:57:26.967950+0000","last_fullsized":"2024-09-17T22:57:26.967950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:05.076128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081202100000000001,"stat_sum":{"num_bytes":131072,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4,"num_read_kb":0,"num_write":4,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:25.963482+0000","last_change":"2024-09-17T22:57:12.701116+0000","last_active":"2024-09-17T22:57:25.963482+0000","last_peered":"2024-09-17T22:57:25.963482+0000","last_clean":"2024-09-17T22:57:25.963482+0000","last_became_active":"2024-09-17T22:57:09.891099+0000","last_became_peered":"2024-09-17T22:57:09.891099+0000","last_unstale":"2024-09-17T22:57:25.963482+0000","last_undegraded":"2024-09-17T22:57:25.963482+0000","last_fullsized":"2024-09-17T22:57:25.963482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:28:35.095994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041524999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"27'3","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.038279+0000","last_change":"2024-09-17T22:57:21.845112+0000","last_active":"2024-09-17T22:57:27.038279+0000","last_peered":"2024-09-17T22:57:27.038279+0000","last_clean":"2024-09-17T22:57:27.038279+0000","last_became_active":"2024-09-17T22:57:19.849300+0000","last_became_peered":"2024-09-17T22:57:19.849300+0000","last_unstale":"2024-09-17T22:57:27.038279+0000","last_undegraded":"2024-09-17T22:57:27.038279+0000","last_fullsized":"2024-09-17T22:57:27.038279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:19:39.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057890600000000002,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.986202+0000","last_change":"2024-09-17T22:57:11.899932+0000","last_active":"2024-09-17T22:57:27.986202+0000","last_peered":"2024-09-17T22:57:27.986202+0000","last_clean":"2024-09-17T22:57:27.986202+0000","last_became_active":"2024-09-17T22:57:09.890192+0000","last_became_peered":"2024-09-17T22:57:09.890192+0000","last_unstale":"2024-09-17T22:57:27.986202+0000","last_undegraded":"2024-09-17T22:57:27.986202+0000","last_fullsized":"2024-09-17T22:57:27.986202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:45.889953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038026899999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":184,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:25.963591+0000","last_change":"2024-09-17T22:57:04.174908+0000","last_active":"2024-09-17T22:57:25.963591+0000","last_peered":"2024-09-17T22:57:25.963591+0000","last_clean":"2024-09-17T22:57:25.963591+0000","last_became_active":"2024-09-17T22:57:04.173923+0000","last_became_peered":"2024-09-17T22:57:04.173923+0000","last_unstale":"2024-09-17T22:57:25.963591+0000","last_undegraded":"2024-09-17T22:57:25.963591+0000","last_fullsized":"2024-09-17T22:57:25.963591+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:23:05.023382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'15","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.984205+0000","last_change":"2024-09-17T22:57:21.845615+0000","last_active":"2024-09-17T22:57:26.984205+0000","last_peered":"2024-09-17T22:57:26.984205+0000","last_clean":"2024-09-17T22:57:26.984205+0000","last_became_active":"2024-09-17T22:57:19.848255+0000","last_became_peered":"2024-09-17T22:57:19.848255+0000","last_unstale":"2024-09-17T22:57:26.984205+0000","last_undegraded":"2024-09-17T22:57:26.984205+0000","last_fullsized":"2024-09-17T22:57:26.984205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:55:01.077231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083459400000000003,"stat_sum":{"num_bytes":29927635,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":87,"num_read_kb":219,"num_write":13,"num_write_kb":225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"28'10","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:27.980489+0000","last_change":"2024-09-17T22:57:21.845736+0000","last_active":"2024-09-17T22:57:27.980489+0000","last_peered":"2024-09-17T22:57:27.980489+0000","last_clean":"2024-09-17T22:57:27.980489+0000","last_became_active":"2024-09-17T22:57:19.951433+0000","last_became_peered":"2024-09-17T22:57:19.951433+0000","last_unstale":"2024-09-17T22:57:27.980489+0000","last_undegraded":"2024-09-17T22:57:27.980489+0000","last_fullsized":"2024-09-17T22:57:27.980489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:23:07.931173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045397599999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4,"num_read_kb":0,"num_write":6,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"30'24","reported_seq":56,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.029372+0000","last_change":"2024-09-17T22:57:22.374427+0000","last_active":"2024-09-17T22:57:28.029372+0000","last_peered":"2024-09-17T22:57:28.029372+0000","last_clean":"2024-09-17T22:57:28.029372+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:57:28.029372+0000","last_undegraded":"2024-09-17T22:57:28.029372+0000","last_fullsized":"2024-09-17T22:57:28.029372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037622699999999997,"stat_sum":{"num_bytes":534392,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":12,"num_read_kb":5,"num_write":18,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"28'7","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.975670+0000","last_change":"2024-09-17T22:57:21.845067+0000","last_active":"2024-09-17T22:57:26.975670+0000","last_peered":"2024-09-17T22:57:26.975670+0000","last_clean":"2024-09-17T22:57:26.975670+0000","last_became_active":"2024-09-17T22:57:19.848467+0000","last_became_peered":"2024-09-17T22:57:19.848467+0000","last_unstale":"2024-09-17T22:57:26.975670+0000","last_undegraded":"2024-09-17T22:57:26.975670+0000","last_fullsized":"2024-09-17T22:57:26.975670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:19:17.736202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056076400000000005,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":11,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"30'11","reported_seq":109,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.189878+0000","last_change":"2024-09-17T22:57:21.851244+0000","last_active":"2024-09-17T22:57:28.189878+0000","last_peered":"2024-09-17T22:57:28.189878+0000","last_clean":"2024-09-17T22:57:28.189878+0000","last_became_active":"2024-09-17T22:57:19.848596+0000","last_became_peered":"2024-09-17T22:57:19.848596+0000","last_unstale":"2024-09-17T22:57:28.189878+0000","last_undegraded":"2024-09-17T22:57:28.189878+0000","last_fullsized":"2024-09-17T22:57:28.189878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:29.891283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087459500000000004,"stat_sum":{"num_bytes":4301151,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":69,"num_read_kb":66,"num_write":6,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"28'4","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.104767+0000","last_change":"2024-09-17T22:57:22.703601+0000","last_active":"2024-09-17T22:57:26.104767+0000","last_peered":"2024-09-17T22:57:26.104767+0000","last_clean":"2024-09-17T22:57:26.104767+0000","last_became_active":"2024-09-17T22:57:19.849680+0000","last_became_peered":"2024-09-17T22:57:19.849680+0000","last_unstale":"2024-09-17T22:57:26.104767+0000","last_undegraded":"2024-09-17T22:57:26.104767+0000","last_fullsized":"2024-09-17T22:57:26.104767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:33.576642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069871699999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":3,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"28'6","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.025624+0000","last_change":"2024-09-17T22:57:22.703669+0000","last_active":"2024-09-17T22:57:26.025624+0000","last_peered":"2024-09-17T22:57:26.025624+0000","last_clean":"2024-09-17T22:57:26.025624+0000","last_became_active":"2024-09-17T22:57:19.851650+0000","last_became_peered":"2024-09-17T22:57:19.851650+0000","last_unstale":"2024-09-17T22:57:26.025624+0000","last_undegraded":"2024-09-17T22:57:26.025624+0000","last_fullsized":"2024-09-17T22:57:26.025624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:14:55.201064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077220800000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"28'6","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.092948+0000","last_change":"2024-09-17T22:57:22.703957+0000","last_active":"2024-09-17T22:57:26.092948+0000","last_peered":"2024-09-17T22:57:26.092948+0000","last_clean":"2024-09-17T22:57:26.092948+0000","last_became_active":"2024-09-17T22:57:19.851755+0000","last_became_peered":"2024-09-17T22:57:19.851755+0000","last_unstale":"2024-09-17T22:57:26.092948+0000","last_undegraded":"2024-09-17T22:57:26.092948+0000","last_fullsized":"2024-09-17T22:57:26.092948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:04.779186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010425580000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"29'17","reported_seq":53,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.121053+0000","last_change":"2024-09-17T22:57:22.374428+0000","last_active":"2024-09-17T22:57:28.121053+0000","last_peered":"2024-09-17T22:57:28.121053+0000","last_clean":"2024-09-17T22:57:28.121053+0000","last_became_active":"2024-09-17T22:57:19.853848+0000","last_became_peered":"2024-09-17T22:57:19.853848+0000","last_unstale":"2024-09-17T22:57:28.121053+0000","last_undegraded":"2024-09-17T22:57:28.121053+0000","last_fullsized":"2024-09-17T22:57:28.121053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:09.886518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036935199999999998,"stat_sum":{"num_bytes":10495411,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":13,"num_read_kb":95,"num_write":12,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"30'11","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T22:57:28.209832+0000","last_change":"2024-09-17T22:57:26.980079+0000","last_active":"2024-09-17T22:57:28.209832+0000","last_peered":"2024-09-17T22:57:28.209832+0000","last_clean":"2024-09-17T22:57:28.209832+0000","last_became_active":"2024-09-17T22:57:26.971893+0000","last_became_peered":"2024-09-17T22:57:26.971893+0000","last_unstale":"2024-09-17T22:57:28.209832+0000","last_undegraded":"2024-09-17T22:57:28.209832+0000","last_fullsized":"2024-09-17T22:57:28.209832+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:52.162384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047249800000000002,"stat_sum":{"num_bytes":294114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5,"num_read_kb":0,"num_write":8,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"28'17","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.967920+0000","last_change":"2024-09-17T22:57:21.855983+0000","last_active":"2024-09-17T22:57:26.967920+0000","last_peered":"2024-09-17T22:57:26.967920+0000","last_clean":"2024-09-17T22:57:26.967920+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:57:26.967920+0000","last_undegraded":"2024-09-17T22:57:26.967920+0000","last_fullsized":"2024-09-17T22:57:26.967920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:03.093554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001538125,"stat_sum":{"num_bytes":19922944,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":13,"num_read_kb":91,"num_write":13,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"28'20","reported_seq":51,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T22:57:26.092670+0000","last_change":"2024-09-17T22:57:22.703350+0000","last_active":"2024-09-17T22:57:26.092670+0000","last_peered":"2024-09-17T22:57:26.092670+0000","last_clean":"2024-09-17T22:57:26.092670+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:57:26.092670+0000","last_undegraded":"2024-09-17T22:57:26.092670+0000","last_fullsized":"2024-09-17T22:57:26.092670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:25.829805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053712699999999998,"stat_sum":{"num_bytes":19931456,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":12,"num_read_kb":7,"num_write":14,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":169665349,"num_objects":144,"num_object_clones":11,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":95,"num_read":2904,"num_read_kb":2918,"num_write":497,"num_write_kb":2438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2576384,"data_stored":2290755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":525,"ondisk_log_size":525,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":313360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":462139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":297388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":419826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":114525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":425984,"data_stored":222973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1224704,"data_stored":990992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":932081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":280277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":108445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":305317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":686909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":467083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":209270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":233438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:57:31.775 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [6, 3] 2024-09-17T22:57:31.775 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.9', '6', '3'] 2024-09-17T22:57:31.775 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.9 6 3 2024-09-17T22:57:31.776 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:31.767+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:31.870 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:31.868+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:31.881 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:31.881 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:31.881 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:31.888 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:31.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:31.973 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:31.971+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:32.024 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:32.024 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:32.034 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:32.035 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:32.035 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:32.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:32.061 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:32.062 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:32.073 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone7 from 0xc2f423c (overlap 0x52620e) to 0x0 2024-09-17T22:57:32.081 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:32.081 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:32.085 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:32.083+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:32.100 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:32.100 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:32.124 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:32.124 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:32.141 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:32.141 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:32.149 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T22:57:32.158 INFO:teuthology.orchestra.run.smithi159.stdout:638 writesame 0x57a7726 thru 0x57a83a4 data_size 0x1(0xc7f bytes) 2024-09-17T22:57:32.159 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:32.159 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:32.174 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:32.171+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:32.175 INFO:teuthology.orchestra.run.smithi159.stdout:639 trunc from 0xc2f423c to 0xa40a73c 2024-09-17T22:57:32.180 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:32.180 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:32.195 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.195 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.205 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:32.212 INFO:teuthology.orchestra.run.smithi159.stdout:640 write 0x44a6a1 thru 0x453f7d (0x98dd bytes) 2024-09-17T22:57:32.263 INFO:teuthology.orchestra.run.smithi159.stdout:641 trunc from 0xa40a73c to 0x22d44ec 2024-09-17T22:57:32.274 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:32.272+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:32.312 INFO:teuthology.orchestra.run.smithi159.stdout:642 writesame 0x6d506dc thru 0x6d5e5fb data_size 0x30(0xdf20 bytes) 2024-09-17T22:57:32.320 INFO:teuthology.orchestra.run.smithi159.stdout:644 write 0x86b35ad thru 0x86b69ae (0x3402 bytes) 2024-09-17T22:57:32.329 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.329 INFO:teuthology.orchestra.run.smithi159.stdout:647 punch from 0x55cc388 to 0x55d6f6e, (0xabe6 bytes) 2024-09-17T22:57:32.329 INFO:teuthology.orchestra.run.smithi159.stdout:648 punch from 0x1e5b818 to 0x1e5fddd, (0x45c5 bytes) 2024-09-17T22:57:32.334 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.334 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.334 INFO:teuthology.orchestra.run.smithi159.stdout:649 trunc from 0x86b69af to 0x39265c2 2024-09-17T22:57:32.344 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:32.356 INFO:teuthology.orchestra.run.smithi159.stdout:650 trunc from 0x39265c2 to 0xaebf604 2024-09-17T22:57:32.362 INFO:teuthology.orchestra.run.smithi159.stdout:651 write 0x2623ef7 thru 0x262a8ab (0x69b5 bytes) 2024-09-17T22:57:32.371 INFO:teuthology.orchestra.run.smithi159.stdout:653 read 0x8abb14d thru 0x8abf5ae (0x4462 bytes) 2024-09-17T22:57:32.371 INFO:teuthology.orchestra.run.smithi159.stdout:656 read 0xa8cf847 thru 0xa8d8728 (0x8ee2 bytes) 2024-09-17T22:57:32.375 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:32.373+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:32.376 INFO:teuthology.orchestra.run.smithi159.stdout:658 write 0xb1d441d thru 0xb1e19cf (0xd5b3 bytes) 2024-09-17T22:57:32.384 INFO:teuthology.orchestra.run.smithi159.stdout:659 write 0x2131fad thru 0x213a4af (0x8503 bytes) 2024-09-17T22:57:32.409 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.409 INFO:teuthology.orchestra.run.smithi159.stdout:661 punch from 0x6014887 to 0x6020c74, (0xc3ed bytes) 2024-09-17T22:57:32.409 INFO:teuthology.orchestra.run.smithi159.stdout:662 read 0xcedf1a thru 0xcef438 (0x151f bytes) 2024-09-17T22:57:32.411 INFO:teuthology.orchestra.run.smithi159.stdout:663 trunc from 0xb1e19d0 to 0xc33eb34 2024-09-17T22:57:32.418 INFO:teuthology.orchestra.run.smithi159.stdout:664 write 0x1548874 thru 0x154bb50 (0x32dd bytes) 2024-09-17T22:57:32.439 INFO:teuthology.orchestra.run.smithi159.stdout:665 trunc from 0xc33eb34 to 0x1b34b42 2024-09-17T22:57:32.476 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:32.474+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:32.478 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.479 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.490 INFO:teuthology.orchestra.run.smithi159.stdout:666 write 0x4d65192 thru 0x4d70ea9 (0xbd18 bytes) 2024-09-17T22:57:32.491 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:32.496 INFO:teuthology.orchestra.run.smithi159.stdout:667 read 0x44861d9 thru 0x448ff3d (0x9d65 bytes) 2024-09-17T22:57:32.496 INFO:teuthology.orchestra.run.smithi159.stdout:668 write 0xd111ac thru 0xd1baa9 (0xa8fe bytes) 2024-09-17T22:57:32.522 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.531 INFO:teuthology.orchestra.run.smithi159.stdout:673 writesame 0x7946858 thru 0x794e6ed data_size 0x21(0x7e96 bytes) 2024-09-17T22:57:32.531 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.531 INFO:teuthology.orchestra.run.smithi159.stdout:677 punch from 0x4b8b0e2 to 0x4b92e66, (0x7d84 bytes) 2024-09-17T22:57:32.546 INFO:teuthology.orchestra.run.smithi159.stdout:678 writesame 0xe89d285 thru 0xe8ad137 data_size 0x1(0xfeb3 bytes) 2024-09-17T22:57:32.546 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.546 INFO:teuthology.orchestra.run.smithi159.stdout:680 punch from 0xae7d8fb to 0xae82ef0, (0x55f5 bytes) 2024-09-17T22:57:32.546 INFO:teuthology.orchestra.run.smithi159.stdout:681 write 0xb508efb thru 0xb51129b (0x83a1 bytes) 2024-09-17T22:57:32.567 INFO:teuthology.orchestra.run.smithi159.stdout:682 trunc from 0xe8ad138 to 0x32b5d10 2024-09-17T22:57:32.577 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:32.575+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:32.582 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.595 INFO:teuthology.orchestra.run.smithi159.stdout:685 trunc from 0x32b5d10 to 0xdd0f1ac 2024-09-17T22:57:32.600 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.600 INFO:teuthology.orchestra.run.smithi159.stdout:687 writesame 0x33bad60 thru 0x33c68c6 data_size 0x5f(0xbb67 bytes) 2024-09-17T22:57:32.608 INFO:teuthology.orchestra.run.smithi159.stdout:688 write 0x1184058 thru 0x1190414 (0xc3bd bytes) 2024-09-17T22:57:32.619 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.619 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:32.643 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.644 INFO:teuthology.orchestra.run.smithi159.stdout:690 read 0x958b54a thru 0x958df98 (0x2a4f bytes) 2024-09-17T22:57:32.644 INFO:teuthology.orchestra.run.smithi159.stdout:691 read 0x92c6a7d thru 0x92c9c3d (0x31c1 bytes) 2024-09-17T22:57:32.644 INFO:teuthology.orchestra.run.smithi159.stdout:692 punch from 0x4537ca to 0x45a6f8, (0x6f2e bytes) 2024-09-17T22:57:32.644 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.644 INFO:teuthology.orchestra.run.smithi159.stdout:695 write 0x1389019 thru 0x1390a1f (0x7a07 bytes) 2024-09-17T22:57:32.664 INFO:teuthology.orchestra.run.smithi159.stdout:696 read 0xc8514c9 thru 0xc85c674 (0xb1ac bytes) 2024-09-17T22:57:32.664 INFO:teuthology.orchestra.run.smithi159.stdout:699 read 0x3fb07b9 thru 0x3fb2b40 (0x2388 bytes) 2024-09-17T22:57:32.664 INFO:teuthology.orchestra.run.smithi159.stdout:700 read 0x78f4ec5 thru 0x7904c52 (0xfd8e bytes) 2024-09-17T22:57:32.664 INFO:teuthology.orchestra.run.smithi159.stdout:701 read 0x2ede4db thru 0x2ee00c2 (0x1be8 bytes) 2024-09-17T22:57:32.664 INFO:teuthology.orchestra.run.smithi159.stdout:702 write 0xdc1ff59 thru 0xdc24e31 (0x4ed9 bytes) 2024-09-17T22:57:32.669 INFO:teuthology.orchestra.run.smithi159.stdout:703 write 0x6cdef7f thru 0x6ceb1e3 (0xc265 bytes) 2024-09-17T22:57:32.678 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:32.675+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:32.681 INFO:teuthology.orchestra.run.smithi159.stdout:705 writesame 0x36f30a1 thru 0x3701916 data_size 0x37(0xe876 bytes) 2024-09-17T22:57:32.695 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.695 INFO:teuthology.orchestra.run.smithi159.stdout:707 writesame 0xd312547 thru 0xd31cefe data_size 0x4(0xa9b8 bytes) 2024-09-17T22:57:32.705 INFO:teuthology.orchestra.run.smithi159.stdout:708 read 0x147223e thru 0x1476b28 (0x48eb bytes) 2024-09-17T22:57:32.705 INFO:teuthology.orchestra.run.smithi159.stdout:710 write 0x96602a7 thru 0x9668dab (0x8b05 bytes) 2024-09-17T22:57:32.711 INFO:teuthology.orchestra.run.smithi159.stdout:711 write 0x53aed12 thru 0x53b3d1b (0x500a bytes) 2024-09-17T22:57:32.718 INFO:teuthology.orchestra.run.smithi159.stdout:712 write 0xdc91205 thru 0xdc9bbf5 (0xa9f1 bytes) 2024-09-17T22:57:32.725 INFO:teuthology.orchestra.run.smithi159.stdout:713 read 0xa57a301 thru 0xa581383 (0x7083 bytes) 2024-09-17T22:57:32.730 INFO:teuthology.orchestra.run.smithi159.stdout:715 trunc from 0xdd0f1ac to 0xa55d046 2024-09-17T22:57:32.741 INFO:teuthology.orchestra.run.smithi159.stdout:716 read 0x5c4c491 thru 0x5c5512f (0x8c9f bytes) 2024-09-17T22:57:32.741 INFO:teuthology.orchestra.run.smithi159.stdout:720 read 0x9f9d057 thru 0x9fa8d4e (0xbcf8 bytes) 2024-09-17T22:57:32.744 INFO:teuthology.orchestra.run.smithi159.stdout:721 trunc from 0xa55d046 to 0x829b837 2024-09-17T22:57:32.762 INFO:teuthology.orchestra.run.smithi159.stdout:722 trunc from 0x829b837 to 0x1ed974f 2024-09-17T22:57:32.762 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.763 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.774 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:32.775 INFO:teuthology.orchestra.run.smithi159.stdout:723 read 0xdea89b thru 0xdf8d5c (0xe4c2 bytes) 2024-09-17T22:57:32.777 INFO:teuthology.orchestra.run.smithi159.stdout:724 punch from 0x7a9af7 to 0x7b331d, (0x9826 bytes) 2024-09-17T22:57:32.777 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:32.790 INFO:teuthology.orchestra.run.smithi159.stdout:727 writesame 0x8668a24 thru 0x86738d0 data_size 0x1(0xaead bytes) 2024-09-17T22:57:32.790 INFO:teuthology.orchestra.run.smithi159.stdout:728 write 0x443bce4 thru 0x443f52c (0x3849 bytes) 2024-09-17T22:57:32.790 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:32.787+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:32.797 INFO:teuthology.orchestra.run.smithi159.stdout:729 write 0x170824f thru 0x17087b2 (0x564 bytes) 2024-09-17T22:57:32.806 INFO:teuthology.orchestra.run.smithi159.stdout:730 trunc from 0x86738d1 to 0x9ae2863 2024-09-17T22:57:32.818 INFO:teuthology.orchestra.run.smithi159.stdout:731 write 0xa56867b thru 0xa56aa60 (0x23e6 bytes) 2024-09-17T22:57:32.825 INFO:teuthology.orchestra.run.smithi159.stdout:732 read 0x55e0bd8 thru 0x55f0463 (0xf88c bytes) 2024-09-17T22:57:32.825 INFO:teuthology.orchestra.run.smithi159.stdout:733 read 0x817aac1 thru 0x817ebf8 (0x4138 bytes) 2024-09-17T22:57:32.834 INFO:teuthology.orchestra.run.smithi159.stdout:734 write 0xd5ffa5f thru 0xd60bf50 (0xc4f2 bytes) 2024-09-17T22:57:32.851 INFO:teuthology.orchestra.run.smithi159.stdout:735 trunc from 0xd60bf51 to 0x9c844c8 2024-09-17T22:57:32.862 INFO:teuthology.orchestra.run.smithi159.stdout:736 clone 9 order 18 su 16384 sc 4 2024-09-17T22:57:32.879 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:32.877+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:32.898 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:32.983 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:32.980+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:33.001 INFO:teuthology.orchestra.run.smithi159.stderr:set 3.9 pg_upmap mapping to [6,3] 2024-09-17T22:57:33.009 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:57:33.051 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.051 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.061 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:33.081 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:33.079+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:33.089 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone8 from 0x9c844c8 (overlap 0x1b34b42) to 0xd9a5a1 2024-09-17T22:57:33.183 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:33.180+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:33.238 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.238 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.247 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:33.287 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:33.285+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:33.385 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:33.383+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:33.390 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.390 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.399 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:33.486 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:33.484+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:33.504 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.504 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:33.587 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:33.584+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:33.617 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.618 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:33.687 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:33.685+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:33.770 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.770 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.779 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:33.823 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:33.821+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:33.881 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:33.882 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:33.889 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:33.886+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:33.890 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:33.990 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:33.988+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:34.000 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:34.001 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:34.001 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:57:34.011 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:34.096 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:34.094+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:34.122 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T22:57:34.123 INFO:teuthology.orchestra.run.smithi159.stdout:738 read 0x1910dec thru 0x191b1c6 (0xa3db bytes) 2024-09-17T22:57:34.126 INFO:teuthology.orchestra.run.smithi159.stdout:741 trunc from 0x9c844c8 to 0x866f93c 2024-09-17T22:57:34.139 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.139 INFO:teuthology.orchestra.run.smithi159.stdout:745 read 0x20dcd55 thru 0x20e8a30 (0xbcdc bytes) 2024-09-17T22:57:34.139 INFO:teuthology.orchestra.run.smithi159.stdout:746 read 0x522b446 thru 0x52396c7 (0xe282 bytes) 2024-09-17T22:57:34.139 INFO:teuthology.orchestra.run.smithi159.stdout:747 writesame 0x6b05b8 thru 0x6b8e2d data_size 0x1(0x8876 bytes) 2024-09-17T22:57:34.162 INFO:teuthology.orchestra.run.smithi159.stdout:748 write 0x1cfff4f thru 0x1d070b3 (0x7165 bytes) 2024-09-17T22:57:34.167 INFO:teuthology.orchestra.run.smithi159.stdout:751 read 0x4bbbe10 thru 0x4bbee53 (0x3044 bytes) 2024-09-17T22:57:34.167 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:34.168 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:34.176 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:34.180 INFO:teuthology.orchestra.run.smithi159.stdout:753 writesame 0xe722d0f thru 0xe72b901 data_size 0x1(0x8bf3 bytes) 2024-09-17T22:57:34.180 INFO:teuthology.orchestra.run.smithi159.stdout:754 read 0xa201803 thru 0xa20459f (0x2d9d bytes) 2024-09-17T22:57:34.180 INFO:teuthology.orchestra.run.smithi159.stdout:755 writesame 0x6c76cdb thru 0x6c82331 data_size 0x1(0xb657 bytes) 2024-09-17T22:57:34.192 INFO:teuthology.orchestra.run.smithi159.stdout:757 read 0xa138a26 thru 0xa13d5bd (0x4b98 bytes) 2024-09-17T22:57:34.192 INFO:teuthology.orchestra.run.smithi159.stdout:758 write 0xd574011 thru 0xd583ff9 (0xffe9 bytes) 2024-09-17T22:57:34.193 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:34.197 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.197 INFO:teuthology.orchestra.run.smithi159.stdout:760 write 0x8160d44 thru 0x816e706 (0xd9c3 bytes) 2024-09-17T22:57:34.205 INFO:teuthology.orchestra.run.smithi159.stdout:761 writesame 0x15d3f6e thru 0x15dcab3 data_size 0x1(0x8b46 bytes) 2024-09-17T22:57:34.218 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:34.216+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:34.233 INFO:teuthology.orchestra.run.smithi159.stdout:762 read 0x3b4337f thru 0x3b4973a (0x63bc bytes) 2024-09-17T22:57:34.244 INFO:teuthology.orchestra.run.smithi159.stdout:763 trunc from 0xe72b902 to 0x5479306 2024-09-17T22:57:34.278 INFO:teuthology.orchestra.run.smithi159.stdout:765 write 0xcc60269 thru 0xcc6773b (0x74d3 bytes) 2024-09-17T22:57:34.284 INFO:teuthology.orchestra.run.smithi159.stdout:767 write 0x5688a3c thru 0x5693928 (0xaeed bytes) 2024-09-17T22:57:34.292 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:34.290+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:34.293 INFO:teuthology.orchestra.run.smithi159.stdout:769 punch from 0xac0bacd to 0xac1a3f0, (0xe923 bytes) 2024-09-17T22:57:34.293 INFO:teuthology.orchestra.run.smithi159.stdout:770 write 0x8bc9b62 thru 0x8bd74f6 (0xd995 bytes) 2024-09-17T22:57:34.302 INFO:teuthology.orchestra.run.smithi159.stdout:771 write 0x18cd711 thru 0x18d5ecc (0x87bc bytes) 2024-09-17T22:57:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:773 writesame 0x7355dac thru 0x735bd42 data_size 0x3(0x5f97 bytes) 2024-09-17T22:57:34.329 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.329 INFO:teuthology.orchestra.run.smithi159.stdout:776 write 0x9a3efc4 thru 0x9a4b5ce (0xc60b bytes) 2024-09-17T22:57:34.329 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:34.330 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:34.337 INFO:teuthology.orchestra.run.smithi159.stdout:777 punch from 0x23d044b to 0x23d101d, (0xbd2 bytes) 2024-09-17T22:57:34.337 INFO:teuthology.orchestra.run.smithi159.stdout:779 read 0x15a1c thru 0x25a0c (0xfff1 bytes) 2024-09-17T22:57:34.338 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:34.338 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:34.339 INFO:teuthology.orchestra.run.smithi159.stdout:780 read 0x3d16e56 thru 0x3d17338 (0x4e3 bytes) 2024-09-17T22:57:34.339 INFO:teuthology.orchestra.run.smithi159.stdout:781 writesame 0x8838c14 thru 0x883aa74 data_size 0xb(0x1e61 bytes) 2024-09-17T22:57:34.340 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:34.348 INFO:teuthology.orchestra.run.smithi159.stdout:782 writesame 0xd8fecf3 thru 0xd90637e data_size 0x36(0x768c bytes) 2024-09-17T22:57:34.348 INFO:teuthology.orchestra.run.smithi159.stdout:783 write 0x703fab3 thru 0x704df86 (0xe4d4 bytes) 2024-09-17T22:57:34.358 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:34.359 INFO:teuthology.orchestra.run.smithi159.stdout:784 write 0xd103e3f thru 0xd10716f (0x3331 bytes) 2024-09-17T22:57:34.366 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.366 INFO:teuthology.orchestra.run.smithi159.stdout:787 writesame 0x39e6e96 thru 0x39f5ab3 data_size 0x1(0xec1e bytes) 2024-09-17T22:57:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:34.376 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:34.382 INFO:teuthology.orchestra.run.smithi159.stdout:788 write 0x48b596a thru 0x48b5e90 (0x527 bytes) 2024-09-17T22:57:34.385 INFO:teuthology.orchestra.run.smithi159.stdout:790 writesame 0x94af2 thru 0x9ca89 data_size 0xc(0x7f98 bytes) 2024-09-17T22:57:34.393 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:34.391+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:34.399 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:34.400 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:34.406 INFO:teuthology.orchestra.run.smithi159.stdout:791 read 0x89d4b46 thru 0x89d7b8a (0x3045 bytes) 2024-09-17T22:57:34.407 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.407 INFO:teuthology.orchestra.run.smithi159.stdout:794 read 0x2a4b30 thru 0x2a7dbe (0x328f bytes) 2024-09-17T22:57:34.407 INFO:teuthology.orchestra.run.smithi159.stdout:795 read 0x6e13222 thru 0x6e1b7e8 (0x85c7 bytes) 2024-09-17T22:57:34.407 INFO:teuthology.orchestra.run.smithi159.stdout:797 read 0x61b44ca thru 0x61b4c69 (0x7a0 bytes) 2024-09-17T22:57:34.408 INFO:teuthology.orchestra.run.smithi159.stdout:798 writesame 0x97af186 thru 0x97b085c data_size 0x1(0x16d7 bytes) 2024-09-17T22:57:34.411 INFO:teuthology.orchestra.run.smithi159.stdout:799 write 0x6b0542b thru 0x6b1528f (0xfe65 bytes) 2024-09-17T22:57:34.417 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.417 INFO:teuthology.orchestra.run.smithi159.stdout:801 write 0x1f0bbc0 thru 0x1f10b4c (0x4f8d bytes) 2024-09-17T22:57:34.421 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:34.421 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:34.421 INFO:teuthology.orchestra.run.smithi159.stdout:802 read 0xa40cc99 thru 0xa41c73b (0xfaa3 bytes) 2024-09-17T22:57:34.421 INFO:teuthology.orchestra.run.smithi159.stdout:804 write 0x4bc5a2f thru 0x4bc85c8 (0x2b9a bytes) 2024-09-17T22:57:34.427 INFO:teuthology.orchestra.run.smithi159.stdout:805 punch from 0xb8691ec to 0xb86f6f9, (0x650d bytes) 2024-09-17T22:57:34.427 INFO:teuthology.orchestra.run.smithi159.stdout:807 read 0xa4fcd34 thru 0xa50828f (0xb55c bytes) 2024-09-17T22:57:34.427 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.427 INFO:teuthology.orchestra.run.smithi159.stdout:810 write 0x66cd18d thru 0x66d0f28 (0x3d9c bytes) 2024-09-17T22:57:34.430 INFO:teuthology.orchestra.run.smithi159.stdout:813 punch from 0x8a55491 to 0x8a575d9, (0x2148 bytes) 2024-09-17T22:57:34.430 INFO:teuthology.orchestra.run.smithi159.stdout:814 read 0x92220ce thru 0x9231c72 (0xfba5 bytes) 2024-09-17T22:57:34.440 INFO:teuthology.orchestra.run.smithi159.stdout:815 trunc from 0xd90637f to 0x5016870 2024-09-17T22:57:34.440 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:34.441 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:34.457 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:34.457 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:34.457 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:34.457 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:34.460 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:34.461 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:34.463 INFO:teuthology.orchestra.run.smithi159.stdout:816 punch from 0x28d501d to 0x28e091a, (0xb8fd bytes) 2024-09-17T22:57:34.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:34.473 INFO:teuthology.orchestra.run.smithi159.stdout:818 trunc from 0x5016870 to 0xcf67281 2024-09-17T22:57:34.478 INFO:teuthology.orchestra.run.smithi159.stdout:819 writesame 0x54c8ed6 thru 0x54cf675 data_size 0x8(0x67a0 bytes) 2024-09-17T22:57:34.479 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:34.479 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:34.482 INFO:teuthology.orchestra.run.smithi159.stdout:820 read 0x19ffc03 thru 0x1a04694 (0x4a92 bytes) 2024-09-17T22:57:34.483 INFO:teuthology.orchestra.run.smithi159.stdout:822 read 0x94a0cc7 thru 0x94a7c22 (0x6f5c bytes) 2024-09-17T22:57:34.483 INFO:teuthology.orchestra.run.smithi159.stdout:823 write 0xa22c629 thru 0xa2398fd (0xd2d5 bytes) 2024-09-17T22:57:34.488 INFO:teuthology.orchestra.run.smithi159.stdout:824 read 0x2576a5f thru 0x257cd01 (0x62a3 bytes) 2024-09-17T22:57:34.489 INFO:teuthology.orchestra.run.smithi159.stdout:825 punch from 0x5e4e6e8 to 0x5e53551, (0x4e69 bytes) 2024-09-17T22:57:34.492 INFO:teuthology.orchestra.run.smithi159.stdout:827 trunc from 0xcf67281 to 0x9e58f88 2024-09-17T22:57:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:828 writesame 0x1af3259 thru 0x1afb2d0 data_size 0x4(0x8078 bytes) 2024-09-17T22:57:34.511 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:34.509+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:34.530 INFO:teuthology.orchestra.run.smithi159.stdout:829 trunc from 0x9e58f88 to 0xcdcf01f 2024-09-17T22:57:34.535 INFO:teuthology.orchestra.run.smithi159.stdout:830 read 0x898b620 thru 0x898d52e (0x1f0f bytes) 2024-09-17T22:57:34.541 INFO:teuthology.orchestra.run.smithi159.stdout:833 write 0xcf86132 thru 0xcf92213 (0xc0e2 bytes) 2024-09-17T22:57:34.545 INFO:teuthology.orchestra.run.smithi159.stdout:834 write 0x13bff8b thru 0x13c055b (0x5d1 bytes) 2024-09-17T22:57:34.557 INFO:teuthology.orchestra.run.smithi159.stdout:835 write 0x965f69 thru 0x96c404 (0x649c bytes) 2024-09-17T22:57:34.574 INFO:teuthology.orchestra.run.smithi159.stdout:837 write 0xcc7b90d thru 0xcc897ca (0xdebe bytes) 2024-09-17T22:57:34.580 INFO:teuthology.orchestra.run.smithi159.stdout:838 read 0x755167c thru 0x755955d (0x7ee2 bytes) 2024-09-17T22:57:34.580 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.580 INFO:teuthology.orchestra.run.smithi159.stdout:845 writesame 0xa250728 thru 0xa2578dc data_size 0x1f(0x71b5 bytes) 2024-09-17T22:57:34.585 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:34.585 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:34.585 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:34.586 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:34.586 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:34.587 INFO:teuthology.orchestra.run.smithi159.stdout:846 read 0x662e1e8 thru 0x66333e6 (0x51ff bytes) 2024-09-17T22:57:34.587 INFO:teuthology.orchestra.run.smithi159.stdout:847 write 0x1089694 thru 0x108f45e (0x5dcb bytes) 2024-09-17T22:57:34.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:34.598 INFO:teuthology.orchestra.run.smithi159.stdout:848 punch from 0x652c96c to 0x6530e35, (0x44c9 bytes) 2024-09-17T22:57:34.603 INFO:teuthology.orchestra.run.smithi159.stdout:849 trunc from 0xcf92214 to 0x876738e 2024-09-17T22:57:34.614 INFO:teuthology.orchestra.run.smithi159.stdout:851 write 0x1a910b8 thru 0x1a9cdc1 (0xbd0a bytes) 2024-09-17T22:57:34.631 INFO:teuthology.orchestra.run.smithi159.stdout:853 writesame 0xb3db125 thru 0xb3e8bfe data_size 0x6d(0xdada bytes) 2024-09-17T22:57:34.631 INFO:teuthology.orchestra.run.smithi159.stdout:854 read 0x3b6799e thru 0x3b70e7d (0x94e0 bytes) 2024-09-17T22:57:34.632 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.632 INFO:teuthology.orchestra.run.smithi159.stdout:856 writesame 0xa72da77 thru 0xa73cf49 data_size 0x6d(0xf4d3 bytes) 2024-09-17T22:57:34.636 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:34.634+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:34.643 INFO:teuthology.orchestra.run.smithi159.stdout:857 writesame 0xd26af45 thru 0xd26ddc3 data_size 0x1(0x2e7f bytes) 2024-09-17T22:57:34.643 INFO:teuthology.orchestra.run.smithi159.stdout:858 read 0xbebc8eb thru 0xbec188b (0x4fa1 bytes) 2024-09-17T22:57:34.643 INFO:teuthology.orchestra.run.smithi159.stdout:859 read 0x491030c thru 0x491aa8f (0xa784 bytes) 2024-09-17T22:57:34.643 INFO:teuthology.orchestra.run.smithi159.stdout:862 write 0xb442adf thru 0xb4508da (0xddfc bytes) 2024-09-17T22:57:34.654 INFO:teuthology.orchestra.run.smithi159.stdout:863 trunc from 0xd26ddc4 to 0x83f09f0 2024-09-17T22:57:34.664 INFO:teuthology.orchestra.run.smithi159.stdout:864 writesame 0x50f4645 thru 0x50fbed9 data_size 0x1(0x7895 bytes) 2024-09-17T22:57:34.671 INFO:teuthology.orchestra.run.smithi159.stdout:866 write 0xcf909e thru 0xcf9b62 (0xac5 bytes) 2024-09-17T22:57:34.676 INFO:teuthology.orchestra.run.smithi159.stdout:867 read 0xe37ae8 thru 0xe38f17 (0x1430 bytes) 2024-09-17T22:57:34.676 INFO:teuthology.orchestra.run.smithi159.stdout:869 read 0x17c1c26 thru 0x17c2dc9 (0x11a4 bytes) 2024-09-17T22:57:34.677 INFO:teuthology.orchestra.run.smithi159.stdout:870 read 0x5c0ab8b thru 0x5c0da9a (0x2f10 bytes) 2024-09-17T22:57:34.677 INFO:teuthology.orchestra.run.smithi159.stdout:871 read 0x3cde0d5 thru 0x3ce61b8 (0x80e4 bytes) 2024-09-17T22:57:34.677 INFO:teuthology.orchestra.run.smithi159.stdout:873 write 0xbd2d77 thru 0xbdb9d0 (0x8c5a bytes) 2024-09-17T22:57:34.697 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:34.694+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:34.701 INFO:teuthology.orchestra.run.smithi159.stdout:874 trunc from 0x83f09f0 to 0xe4e812c 2024-09-17T22:57:34.706 INFO:teuthology.orchestra.run.smithi159.stdout:875 read 0x5fc3c0e thru 0x5fc834a (0x473d bytes) 2024-09-17T22:57:34.712 INFO:teuthology.orchestra.run.smithi159.stdout:876 trunc from 0xe4e812c to 0x9c0172c 2024-09-17T22:57:34.718 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.718 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.730 INFO:teuthology.orchestra.run.smithi159.stdout:880 trunc from 0x9c0172c to 0x2aa962 2024-09-17T22:57:34.750 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:34.750 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1206 3.11 3:8eee0039:::rbd_data.118840f89003.00000000000000bc:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6827734650000004, 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90939317600000003, 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.751 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1206 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064327+0000", 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064328+0000", 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6899999999999997e-07 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064331+0000", 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0589999999999998e-06 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064338+0000", 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6100000000000002e-06 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064342+0000", 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064385+0000", 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2216999999999999e-05 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.064391+0000", 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.117e-06 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972129+0000", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90773784000000002 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972245+0000", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000116396 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973721+0000", 2024-09-17T22:57:34.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001475456 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1210 3.11 3:8af04179:::rbd_data.118840f89003.00000000000000bf:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6801210310000005, 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90700086899999999, 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1210 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066980+0000", 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2499999999999995e-07 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066984+0000", 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7249999999999999e-06 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.066996+0000", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2337000000000001e-05 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067002+0000", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6420000000000004e-06 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067055+0000", 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2797000000000003e-05 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.067064+0000", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0569999999999999e-06 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972638+0000", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90557432699999996 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972737+0000", 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8486999999999994e-05 2024-09-17T22:57:34.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973981+0000", 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012439720000000001 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1235 3.11 3:8a065d9f:::rbd_data.118840f89003.00000000000000c6:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6736093329999999, 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.90162610200000004, 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.762 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1235 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073491+0000", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073492+0000", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2699999999999998e-07 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073497+0000", 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8099999999999997e-06 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073510+0000", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2405000000000001e-05 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073517+0000", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0389999999999999e-06 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073571+0000", 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4159000000000001e-05 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.073580+0000", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0880000000000001e-06 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.972930+0000", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89935040799999999 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973012+0000", 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1601999999999996e-05 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975118+0000", 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021056640000000001 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:1249 3.11 3:89eed2ba:::rbd_data.118840f89003.00000000000000c8:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.6671699229999994, 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89530808699999997, 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1249 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079931+0000", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079932+0000", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5000000000000002e-07 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079941+0000", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7119999999999995e-06 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079953+0000", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1841000000000001e-05 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079961+0000", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5599999999999996e-06 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.079999+0000", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8642e-05 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:34.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.080005+0000", 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2029999999999997e-06 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973323+0000", 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.89331756399999995 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.973409+0000", 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5636999999999993e-05 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:26.975239+0000", 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018302780000000001 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4069 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:34.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3936134779999998, 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012661786, 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4069 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353487+0000", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353488+0000", 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:57:34.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353490+0000", 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8929999999999999e-06 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353495+0000", 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5280000000000003e-06 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353501+0000", 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6509999999999998e-06 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366040+0000", 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012539246 2024-09-17T22:57:34.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366086+0000", 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6369000000000003e-05 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366149+0000", 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2754000000000003e-05 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.777 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4070 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.393593912, 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012774396, 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.778 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4070 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353507+0000", 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8099999999999998e-07 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353509+0000", 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.838e-06 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353515+0000", 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.49e-06 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353526+0000", 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1556000000000001e-05 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366173+0000", 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012646749000000001 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366220+0000", 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7205e-05 2024-09-17T22:57:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366281+0000", 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1177000000000001e-05 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4071 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2473984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:34.783 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3935681300000002, 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012885403, 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4071 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353533+0000", 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353535+0000", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0420000000000001e-06 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353542+0000", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7479999999999996e-06 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353549+0000", 2024-09-17T22:57:34.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9609999999999999e-06 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366309+0000", 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012760102000000001 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366353+0000", 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4535e-05 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366418+0000", 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4627000000000003e-05 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4072 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3935405420000002, 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012992178, 2024-09-17T22:57:34.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4072 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353560+0000", 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353561+0000", 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5200000000000002e-07 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353563+0000", 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1629999999999999e-06 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353569+0000", 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1190000000000002e-06 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353574+0000", 2024-09-17T22:57:34.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.729e-06 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366445+0000", 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012871356 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366492+0000", 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6959000000000002e-05 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366552+0000", 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0399999999999998e-05 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4073 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3935197940000004, 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013104863, 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4073 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353581+0000", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353583+0000", 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9180000000000001e-06 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353589+0000", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5890000000000002e-06 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353595+0000", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.319e-06 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366580+0000", 2024-09-17T22:57:34.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.012984961 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366622+0000", 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2245e-05 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366686+0000", 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3485000000000006e-05 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4074 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.393498342, 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013221811, 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4074 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353602+0000", 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353603+0000", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353605+0000", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9659999999999999e-06 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353611+0000", 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0100000000000001e-06 2024-09-17T22:57:34.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353615+0000", 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4449999999999998e-06 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366716+0000", 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013101218 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366764+0000", 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7447000000000002e-05 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366824+0000", 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0392e-05 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4075 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3934795050000002, 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013339781, 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4075 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353621+0000", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353622+0000", 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353624+0000", 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9700000000000002e-06 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353629+0000", 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.977e-06 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353631+0000", 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366852+0000", 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013220849 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366898+0000", 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5534999999999997e-05 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366961+0000", 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3195999999999996e-05 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4076 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3934590809999996, 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013453582, 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4076 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353642+0000", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8099999999999999e-07 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353644+0000", 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.72e-06 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353650+0000", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.02e-06 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353653+0000", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0769999999999999e-06 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.366988+0000", 2024-09-17T22:57:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013335157 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367036+0000", 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7642999999999998e-05 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367095+0000", 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9684e-05 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4077 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3934381340000002, 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013566762, 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4077 2024-09-17T22:57:34.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:34.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353663+0000", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9400000000000001e-07 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353669+0000", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5350000000000001e-06 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353674+0000", 2024-09-17T22:57:34.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.605e-06 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353682+0000", 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6650000000000003e-06 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367122+0000", 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013440275 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367173+0000", 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1119999999999998e-05 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367229+0000", 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6267999999999997e-05 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.816 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4078 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 3850240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3934149869999999, 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013684187, 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4078 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353686+0000", 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1100000000000002e-07 2024-09-17T22:57:34.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353694+0000", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9950000000000005e-06 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353700+0000", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1879999999999999e-06 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353703+0000", 2024-09-17T22:57:34.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1779999999999999e-06 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367258+0000", 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013554237 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367305+0000", 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7630000000000003e-05 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367370+0000", 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4647999999999995e-05 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4079 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3933867830000004, 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013897985, 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4079 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353714+0000", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8200000000000001e-07 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353716+0000", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6339999999999999e-06 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353720+0000", 2024-09-17T22:57:34.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.721e-06 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353728+0000", 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8590000000000005e-06 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367406+0000", 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013678832 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367474+0000", 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7979999999999996e-05 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367612+0000", 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000137677 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4080 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4374528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:34.826 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3933695229999996, 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014035901999999999, 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4080 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:34.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353731+0000", 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353733+0000", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3200000000000001e-06 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353736+0000", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.326e-06 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353740+0000", 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.653e-06 2024-09-17T22:57:34.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367646+0000", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013906452 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367696+0000", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0105000000000002e-05 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367767+0000", 2024-09-17T22:57:34.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0834000000000007e-05 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4081 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3933580140000004, 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014163906, 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4081 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353743+0000", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353744+0000", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.158e-06 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353747+0000", 2024-09-17T22:57:34.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9840000000000001e-06 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353750+0000", 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.486e-06 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367797+0000", 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014047389 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367848+0000", 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1459999999999999e-05 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367907+0000", 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8220000000000002e-05 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4082 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4767744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3933468930000004, 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014293589000000001, 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4082 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353754+0000", 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353755+0000", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0389999999999999e-06 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353759+0000", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9249999999999997e-06 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353762+0000", 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5040000000000001e-06 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367934+0000", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014172842 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.367983+0000", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8599e-05 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368047+0000", 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4448000000000004e-05 2024-09-17T22:57:34.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4083 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 4964352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.393335456, 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014410081, 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4083 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353765+0000", 2024-09-17T22:57:34.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353766+0000", 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353767+0000", 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0759999999999999e-06 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353771+0000", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9299999999999996e-06 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353773+0000", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3580000000000001e-06 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368075+0000", 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014302265999999999 2024-09-17T22:57:34.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368121+0000", 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5667000000000002e-05 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368175+0000", 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4558999999999997e-05 2024-09-17T22:57:34.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:4084 3.c 3:30443dd6:::rbd_data.1188b3deeabb.000000000000000f:8 [read 5160960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3933247519999998, 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014534003, 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:34.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 4084 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:34.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353776+0000", 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353781+0000", 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1279999999999999e-06 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353785+0000", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5389999999999999e-06 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.353787+0000", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.402e-06 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368196+0000", 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014409065 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368250+0000", 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3699000000000002e-05 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:30.368310+0000", 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9962e-05 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:34.849 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:34.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:34.852 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:34.848+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:34.852 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:886 read 0x12ab3b thru 0x12b914 (0xdda bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:887 writesame 0x7bbdf57 thru 0x7bca9ac data_size 0x61(0xca56 bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:889 trunc from 0x7bca9ad to 0xa6e9756 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:892 write 0x3cdabbb thru 0x3cdde2c (0x3272 bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:894 write 0xbd7429f thru 0xbd7b998 (0x76fa bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:895 punch from 0xe1b14d to 0xe20c80, (0x5b33 bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:897 read 0x8ba942 thru 0x8bb5d9 (0xc98 bytes) 2024-09-17T22:57:34.853 INFO:teuthology.orchestra.run.smithi159.stdout:898 trunc from 0xbd7b999 to 0x411ccb5 2024-09-17T22:57:34.859 INFO:teuthology.orchestra.run.smithi159.stdout:900 write 0xa74bef thru 0xa76ef6 (0x2308 bytes) 2024-09-17T22:57:34.863 INFO:teuthology.orchestra.run.smithi159.stdout:901 read 0x3974999 thru 0x3979d8e (0x53f6 bytes) 2024-09-17T22:57:34.863 INFO:teuthology.orchestra.run.smithi159.stdout:903 write 0x142cf82 thru 0x142f4cb (0x254a bytes) 2024-09-17T22:57:34.865 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.880 INFO:teuthology.orchestra.run.smithi159.stdout:905 writesame 0xb642551 thru 0xb645d0d data_size 0x1(0x37bd bytes) 2024-09-17T22:57:34.880 INFO:teuthology.orchestra.run.smithi159.stdout:906 read 0x655aa55 thru 0x6561406 (0x69b2 bytes) 2024-09-17T22:57:34.880 INFO:teuthology.orchestra.run.smithi159.stdout:907 read 0x24417ec thru 0x244b0e1 (0x98f6 bytes) 2024-09-17T22:57:34.880 INFO:teuthology.orchestra.run.smithi159.stdout:908 read 0x953fe0b thru 0x954b3bf (0xb5b5 bytes) 2024-09-17T22:57:34.880 INFO:teuthology.orchestra.run.smithi159.stdout:909 write 0x9d8fa1 thru 0x9dd11c (0x417c bytes) 2024-09-17T22:57:34.885 INFO:teuthology.orchestra.run.smithi159.stdout:911 read 0x7a1cfd3 thru 0x7a21f64 (0x4f92 bytes) 2024-09-17T22:57:34.885 INFO:teuthology.orchestra.run.smithi159.stdout:912 write 0x59e0d0b thru 0x59e172a (0xa20 bytes) 2024-09-17T22:57:34.893 INFO:teuthology.orchestra.run.smithi159.stdout:913 read 0xfcb07b thru 0xfd5fb4 (0xaf3a bytes) 2024-09-17T22:57:34.893 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.893 INFO:teuthology.orchestra.run.smithi159.stdout:918 writesame 0xb41ec23 thru 0xb428e1f data_size 0x17(0xa1fd bytes) 2024-09-17T22:57:34.906 INFO:teuthology.orchestra.run.smithi159.stdout:919 write 0xd685fd5 thru 0xd68d84d (0x7879 bytes) 2024-09-17T22:57:34.910 INFO:teuthology.orchestra.run.smithi159.stdout:920 read 0x5e56f42 thru 0x5e5d2c2 (0x6381 bytes) 2024-09-17T22:57:34.910 INFO:teuthology.orchestra.run.smithi159.stdout:922 read 0x36dcc3e thru 0x36de348 (0x170b bytes) 2024-09-17T22:57:34.911 INFO:teuthology.orchestra.run.smithi159.stdout:923 read 0x7aa7c5d thru 0x7ab1124 (0x94c8 bytes) 2024-09-17T22:57:34.911 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.911 INFO:teuthology.orchestra.run.smithi159.stdout:926 read 0x3f96099 thru 0x3f9a6b1 (0x4619 bytes) 2024-09-17T22:57:34.911 INFO:teuthology.orchestra.run.smithi159.stdout:927 read 0x211373d thru 0x2119743 (0x6007 bytes) 2024-09-17T22:57:34.915 INFO:teuthology.orchestra.run.smithi159.stdout:928 writesame 0xe00f81d thru 0xe01766e data_size 0x2e(0x7e52 bytes) 2024-09-17T22:57:34.916 INFO:teuthology.orchestra.run.smithi159.stdout:931 punch from 0x759f6c3 to 0x75ac54b, (0xce88 bytes) 2024-09-17T22:57:34.916 INFO:teuthology.orchestra.run.smithi159.stdout:932 write 0xf6b4bc thru 0xf739c6 (0x850b bytes) 2024-09-17T22:57:34.927 INFO:teuthology.orchestra.run.smithi159.stdout:933 trunc from 0xe01766f to 0x7d9b72a 2024-09-17T22:57:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:934 read 0x665c91 thru 0x666e1a (0x118a bytes) 2024-09-17T22:57:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:935 read 0x1533292 thru 0x15411e8 (0xdf57 bytes) 2024-09-17T22:57:34.952 INFO:teuthology.orchestra.run.smithi159.stdout:937 write 0xec7b86c thru 0xec8a237 (0xe9cc bytes) 2024-09-17T22:57:34.952 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:34.949+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:940 write 0x985b4a8 thru 0x985b7e1 (0x33a bytes) 2024-09-17T22:57:34.971 INFO:teuthology.orchestra.run.smithi159.stdout:941 trunc from 0xec8a238 to 0x5100db1 2024-09-17T22:57:34.996 INFO:teuthology.orchestra.run.smithi159.stdout:942 write 0xe9b0c54 thru 0xe9b4f16 (0x42c3 bytes) 2024-09-17T22:57:34.999 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:34.999 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:35.002 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.002 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:35.002 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:35.002 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.002 INFO:teuthology.orchestra.run.smithi159.stdout:947 trunc from 0xe9b4f17 to 0xc6ab985 2024-09-17T22:57:35.011 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:35.016 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:35.016 INFO:teuthology.orchestra.run.smithi159.stdout:950 write 0x9203313 thru 0x92083cd (0x50bb bytes) 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:952 punch from 0x3a0c17a to 0x3a15f1e, (0x9da4 bytes) 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:955 read 0x92718fc thru 0x9277932 (0x6037 bytes) 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:956 read 0x3782ad8 thru 0x378d9d2 (0xaefb bytes) 2024-09-17T22:57:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:957 write 0x87d74b1 thru 0x87e43ff (0xcf4f bytes) 2024-09-17T22:57:35.025 INFO:teuthology.orchestra.run.smithi159.stdout:959 write 0x9ef58e8 thru 0x9efb470 (0x5b89 bytes) 2024-09-17T22:57:35.028 INFO:teuthology.orchestra.run.smithi159.stdout:960 write 0xa30136e thru 0xa30e446 (0xd0d9 bytes) 2024-09-17T22:57:35.032 INFO:teuthology.orchestra.run.smithi159.stdout:961 read 0x74384b6 thru 0x7438ae2 (0x62d bytes) 2024-09-17T22:57:35.038 INFO:teuthology.orchestra.run.smithi159.stdout:962 write 0xd7a75e6 thru 0xd7ad168 (0x5b83 bytes) 2024-09-17T22:57:35.043 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:35.043 INFO:teuthology.orchestra.run.smithi159.stdout:964 write 0x7fd97a3 thru 0x7fe040b (0x6c69 bytes) 2024-09-17T22:57:35.046 INFO:teuthology.orchestra.run.smithi159.stdout:966 write 0x5207d11 thru 0x5213c6a (0xbf5a bytes) 2024-09-17T22:57:35.055 INFO:teuthology.orchestra.run.smithi159.stdout:967 read 0xaea3e4f thru 0xaea62b4 (0x2466 bytes) 2024-09-17T22:57:35.055 INFO:teuthology.orchestra.run.smithi159.stdout:968 write 0x48b5136 thru 0x48bfb36 (0xaa01 bytes) 2024-09-17T22:57:35.055 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:35.053+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:35.062 INFO:teuthology.orchestra.run.smithi159.stdout:969 punch from 0x4d1390f to 0x4d15731, (0x1e22 bytes) 2024-09-17T22:57:35.062 INFO:teuthology.orchestra.run.smithi159.stdout:970 punch from 0xa5069fc to 0xa507632, (0xc36 bytes) 2024-09-17T22:57:35.062 INFO:teuthology.orchestra.run.smithi159.stdout:971 read 0x17d1fd7 thru 0x17d98aa (0x78d4 bytes) 2024-09-17T22:57:35.077 INFO:teuthology.orchestra.run.smithi159.stdout:972 trunc from 0xd7ad169 to 0xa60a6b 2024-09-17T22:57:35.116 INFO:teuthology.orchestra.run.smithi159.stdout:973 writesame 0xce57382 thru 0xce585b0 data_size 0x1(0x122f bytes) 2024-09-17T22:57:35.116 INFO:teuthology.orchestra.run.smithi159.stdout:974 clone 10 order 25 su 524288 sc 13 2024-09-17T22:57:35.123 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:35.123 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:35.123 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.132 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:35.155 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:35.152+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:35.255 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:35.253+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8293 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1048576~81920] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.9706895680000001, 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014439354, 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8293 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303835+0000", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303836+0000", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303837+0000", 2024-09-17T22:57:35.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303849+0000", 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1089e-05 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303854+0000", 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9740000000000001e-06 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.315219+0000", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011365831 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.315321+0000", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101469 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318274+0000", 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029536359999999999 2024-09-17T22:57:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8294 3.10 3:0c66fb5d:::rbd_data.118846a2cbc0.000000000000000a:9 [sparse-read 1654784~81920] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.9706645190000001, 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016892707999999999, 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8294 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303860+0000", 2024-09-17T22:57:35.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303861+0000", 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303867+0000", 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303878+0000", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0560999999999999e-05 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.303883+0000", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318353+0000", 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470304 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.318442+0000", 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9520999999999999e-05 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320753+0000", 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023103939999999999 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.287 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8367 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 262144~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.9701694229999998, 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016773656000000001, 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8367 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304355+0000", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304356+0000", 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0500000000000003e-07 2024-09-17T22:57:35.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304359+0000", 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.501e-06 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304367+0000", 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304381+0000", 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4491000000000001e-05 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320796+0000", 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016414173000000001 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.320859+0000", 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3272e-05 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321129+0000", 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027004600000000002 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8368 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 868352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.969177475, 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015923571000000001, 2024-09-17T22:57:35.292 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8368 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305347+0000", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305348+0000", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9699999999999994e-07 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305353+0000", 2024-09-17T22:57:35.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000003e-06 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305362+0000", 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9279999999999999e-06 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305371+0000", 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6090000000000001e-06 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321157+0000", 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015786517 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321214+0000", 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7021000000000002e-05 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321271+0000", 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6579000000000002e-05 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.296 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8369 3.10 3:0dd5fb06:::rbd_data.118846a2cbc0.000000000000002a:9 [sparse-read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.969057979, 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01592536, 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.297 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8369 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305467+0000", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1799999999999995e-07 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305471+0000", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5889999999999999e-06 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305478+0000", 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5519999999999999e-06 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.305485+0000", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1879999999999999e-06 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321295+0000", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015810267999999999 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321339+0000", 2024-09-17T22:57:35.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4184000000000003e-05 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.321392+0000", 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2961000000000003e-05 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13561 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3186688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504166+0000", 2024-09-17T22:57:35.301 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77035858499999998, 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014179034, 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13561 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504166+0000", 2024-09-17T22:57:35.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504166+0000", 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504166+0000", 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504167+0000", 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1399999999999995e-07 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504170+0000", 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1939999999999998e-06 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504180+0000", 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.859e-06 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518225+0000", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014044536999999999 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518286+0000", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1451000000000005e-05 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518345+0000", 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8851999999999998e-05 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13562 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3579904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504185+0000", 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77033936400000003, 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014306869999999999, 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13562 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504185+0000", 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504185+0000", 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504186+0000", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.67e-07 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504187+0000", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504193+0000", 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5769999999999999e-06 2024-09-17T22:57:35.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504204+0000", 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0682999999999999e-05 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518379+0000", 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014174847000000001 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518428+0000", 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9032000000000001e-05 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518492+0000", 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4190999999999996e-05 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13563 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3776512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504218+0000", 2024-09-17T22:57:35.310 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77030660799999995, 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014514890000000001, 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13563 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504218+0000", 2024-09-17T22:57:35.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504218+0000", 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504219+0000", 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504221+0000", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9910000000000001e-06 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504229+0000", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.123e-06 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504234+0000", 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6160000000000001e-06 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518537+0000", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014302639000000001 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518631+0000", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4220000000000003e-05 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518733+0000", 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010184299999999999 2024-09-17T22:57:35.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13564 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3973120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504244+0000", 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77028059800000004, 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014751673999999999, 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13564 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504244+0000", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504244+0000", 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504244+0000", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504246+0000", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9980000000000002e-06 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504252+0000", 2024-09-17T22:57:35.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504261+0000", 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4109999999999996e-06 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518788+0000", 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014526768000000001 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518882+0000", 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4506000000000006e-05 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.518996+0000", 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011365 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13565 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 4169728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504267+0000", 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77025733200000002, 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01496755, 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13565 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504267+0000", 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504267+0000", 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504268+0000", 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504270+0000", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9700000000000002e-06 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504276+0000", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1550000000000004e-06 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504282+0000", 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7919999999999998e-06 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519031+0000", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014749330999999999 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519124+0000", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2993999999999998e-05 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519235+0000", 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110883 2024-09-17T22:57:35.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13566 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3244032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504290+0000", 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77023489199999995, 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015096895000000001, 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13566 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504290+0000", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504290+0000", 2024-09-17T22:57:35.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504290+0000", 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504292+0000", 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.717e-06 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504299+0000", 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8700000000000003e-06 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504304+0000", 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8300000000000003e-06 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519274+0000", 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014970545 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519324+0000", 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0204999999999998e-05 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519387+0000", 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2316000000000004e-05 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13567 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3637248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504310+0000", 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.770214972, 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015229763, 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13567 2024-09-17T22:57:35.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504310+0000", 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504310+0000", 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504310+0000", 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504312+0000", 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.897e-06 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504319+0000", 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7610000000000002e-06 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504328+0000", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3049999999999998e-06 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519422+0000", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015093883000000001 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519474+0000", 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2309999999999999e-05 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519539+0000", 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5223999999999998e-05 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13568 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3440640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504337+0000", 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77018814999999996, 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015392528000000001, 2024-09-17T22:57:35.333 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13568 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504337+0000", 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504337+0000", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504337+0000", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5699999999999998e-07 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504339+0000", 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.979e-06 2024-09-17T22:57:35.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504346+0000", 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.153e-06 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504352+0000", 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519578+0000", 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015225765 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519653+0000", 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5215000000000004e-05 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519729+0000", 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5859999999999995e-05 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.337 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13569 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3833856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504358+0000", 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77016616900000001, 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015823763000000001, 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13569 2024-09-17T22:57:35.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504358+0000", 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504358+0000", 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504359+0000", 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504365+0000", 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8699999999999997e-06 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504376+0000", 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1110999999999999e-05 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504381+0000", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3340000000000001e-06 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.519972+0000", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015590836 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520069+0000", 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6810000000000003e-05 2024-09-17T22:57:35.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520182+0000", 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113367 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13570 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3252224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504390+0000", 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77013429799999999, 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015916979000000001, 2024-09-17T22:57:35.342 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13570 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504390+0000", 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504390+0000", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504391+0000", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.46e-07 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504399+0000", 2024-09-17T22:57:35.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2719999999999997e-06 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504406+0000", 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3470000000000001e-06 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504420+0000", 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3651e-05 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520201+0000", 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015780921 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520250+0000", 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8702000000000002e-05 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520307+0000", 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7639999999999997e-05 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13571 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 4030464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504443+0000", 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77008192200000003, 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016120266000000001, 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13571 2024-09-17T22:57:35.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504443+0000", 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504443+0000", 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504443+0000", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504445+0000", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504453+0000", 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0879999999999995e-06 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504457+0000", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9779999999999999e-06 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520354+0000", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.015897313 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520453+0000", 2024-09-17T22:57:35.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8148999999999998e-05 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520563+0000", 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110436 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13572 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504462+0000", 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77006230399999998, 2024-09-17T22:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016214309, 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13572 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504462+0000", 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504462+0000", 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504463+0000", 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6300000000000001e-07 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504464+0000", 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2309999999999999e-06 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504469+0000", 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504471+0000", 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.61e-06 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520578+0000", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016106928999999999 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520623+0000", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4736e-05 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520677+0000", 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3835000000000002e-05 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13573 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3842048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504474+0000", 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77005025000000005, 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016446821, 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13573 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504474+0000", 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504474+0000", 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504475+0000", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504476+0000", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1200000000000001e-06 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504479+0000", 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3400000000000002e-06 2024-09-17T22:57:35.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504483+0000", 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520720+0000", 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016237558999999999 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520819+0000", 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8788000000000004e-05 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520921+0000", 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102323 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13574 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 3645440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504486+0000", 2024-09-17T22:57:35.360 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77003895899999997, 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016569243000000001, 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13574 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504486+0000", 2024-09-17T22:57:35.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504486+0000", 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504486+0000", 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2499999999999999e-07 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504487+0000", 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.083e-06 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504494+0000", 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6420000000000001e-06 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504497+0000", 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3129999999999999e-06 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520953+0000", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016455770000000002 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.520999+0000", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6470000000000001e-05 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.521055+0000", 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5739999999999998e-05 2024-09-17T22:57:35.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:13575 3.10 3:099435c5:::rbd_data.1188b0ea165c.0000000000000005:a [read 4038656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:34.504500+0000", 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.77002458299999998, 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016791585000000001, 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 13575 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504500+0000", 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504500+0000", 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504500+0000", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3999999999999998e-07 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504507+0000", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4010000000000002e-06 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504510+0000", 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6859999999999999e-06 2024-09-17T22:57:35.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.504513+0000", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.616e-06 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.521085+0000", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.016572254000000002 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.521184+0000", 2024-09-17T22:57:35.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8308999999999997e-05 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:34.521292+0000", 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010807900000000001 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.369 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.371 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:35.371 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:35.368+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:35.471 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:35.469+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:35.515 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.515 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:35.515 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:35.515 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.524 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:35.572 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:35.569+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:35.627 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.627 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:35.628 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:35.628 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:35.628 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:35.677 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:35.675+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:35.774 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:35.771+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.917181152000001, 2024-09-17T22:57:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:57:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:57:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:57:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6904978120000003, 2024-09-17T22:57:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:57:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.690463276, 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:57:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6904088929999999, 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:57:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6903838889999996, 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:57:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6903633080000002, 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:57:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:57:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:57:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:57:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6903420249999996, 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:57:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6903234989999998, 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6903053269999999, 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:57:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:57:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:57:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:57:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6902874990000001, 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:57:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6899723660000001, 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:57:35.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6892500950000002, 2024-09-17T22:57:35.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:35.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:57:35.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6892054989999998, 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.855 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6891781769999996, 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.856 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:35.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:57:35.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6873593790000001, 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.861 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:57:35.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:57:35.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6870165740000003, 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:57:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:57:35.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6847454419999996, 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:57:35.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:57:35.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.876 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6847133479999998, 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:57:35.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:35.881 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6846868099999996, 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:35.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.6846679169999996, 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:57:35.886 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:57:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:57:35.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:35.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:35.891 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:35.893 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:35.890+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:35.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:35.993 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:35.991+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:36.008 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:36.008 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:36.009 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:36.009 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:36.017 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:36.019 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:57:36.094 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:36.091+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:36.118 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone9 from 0xce585b1 (overlap 0x2aa962) to 0x22211b 2024-09-17T22:57:36.136 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:36.136 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:36.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:36.195 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:36.193+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:36.239 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T22:57:36.296 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:36.293+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:36.327 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059449890999999998, 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.328 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:57:36.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16076 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059432363000000002, 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306234000000001, 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.332 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16076 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:36.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265300+0000", 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2520000000000005e-06 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265308+0000", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2709999999999992e-06 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265313+0000", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8770000000000001e-06 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318592+0000", 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053279435 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318596+0000", 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0389999999999998e-06 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059345855000000003, 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.336 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059324594000000001, 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:57:36.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:57:36.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:36.344 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059303251000000001, 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.348 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059233793999999999, 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:57:36.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:57:36.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.352 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059214816000000003, 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:57:36.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.356 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059185887, 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.357 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:57:36.360 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059159949000000003, 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:57:36.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:57:36.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059131955, 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.365 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:57:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:57:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059111031000000001, 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.369 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:57:36.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059064289999999998, 2024-09-17T22:57:36.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:57:36.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:36.377 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059041881999999997, 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:36.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:57:36.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.382 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.059018538000000002, 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:57:36.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:57:36.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058995849000000003, 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.387 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:36.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058973650000000002, 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.391 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:57:36.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:36.395 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058944864, 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:36.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:57:36.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058922230999999999, 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.400 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:57:36.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058902978000000002, 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:57:36.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.407 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.058875801999999998, 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:36.408 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:36.412 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:36.413 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:36.414 INFO:teuthology.orchestra.run.smithi159.stdout:975 clone 11 order 22 su 262144 sc 5 2024-09-17T22:57:36.414 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:36.411+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:36.492 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:36.514 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:36.512+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:36.559 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:36.560 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:36.560 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:36.560 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:36.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:36.615 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:36.613+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:36.618 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:36.619 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:36.636 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:36.636 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:36.652 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:36.653 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:36.668 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:36.669 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:36.675 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:36.675 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:36.684 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:36.685 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:36.685 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:36.702 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:36.703 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:36.716 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:36.713+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:36.719 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:36.719 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:36.736 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:36.736 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:36.790 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:36.790 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:36.798 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:36.842 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:36.840+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:36.902 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:36.902 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:36.910 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:36.952 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:36.949+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:37.019 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:37.017+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:37.037 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:57:37.058 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.059 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.068 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:37.135 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone10 from 0xce585b1 (overlap 0xce585b1) to 0x14a26f8 2024-09-17T22:57:37.136 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:37.133+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:37.221 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:37.218+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:37.223 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.223 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.233 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:37.292 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T22:57:37.317 INFO:teuthology.orchestra.run.smithi159.stdout:976 clone 12 order 25 su 524288 sc 4 2024-09-17T22:57:37.322 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:37.319+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:37.381 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.381 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:37.423 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:37.420+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:37.524 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:37.521+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:37.534 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.534 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.543 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:37.626 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:37.624+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:37.686 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.686 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.695 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:37.735 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:37.733+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:37.798 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.799 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:37.826 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:37.823+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:37.910 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:37.911 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:37.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:37.963 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:37.960+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:38.019 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:38.020 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:38.020 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T22:57:38.020 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T22:57:38.023 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.023 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.027 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:38.024+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:38.031 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:38.129 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:38.126+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:38.140 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone11 from 0xce585b1 (overlap 0xce585b1) to 0xb9b5eb8 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:38.166 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:38.167 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:38.167 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:38.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:38.169 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:38.170 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T22:57:38.184 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.184 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:38.234 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:38.231+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:38.310 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:38.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:38.314 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:38.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T22:57:38.341 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.342 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:38.358 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:38.356+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:38.390 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T22:57:38.420 INFO:teuthology.orchestra.run.smithi159.stdout:977 punch from 0x6370c02 to 0x63778a0, (0x6c9e bytes) 2024-09-17T22:57:38.424 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.425 INFO:teuthology.orchestra.run.smithi159.stdout:980 read 0x709b554 thru 0x709d1ac (0x1c59 bytes) 2024-09-17T22:57:38.425 INFO:teuthology.orchestra.run.smithi159.stdout:981 read 0x183f706 thru 0x184d673 (0xdf6e bytes) 2024-09-17T22:57:38.425 INFO:teuthology.orchestra.run.smithi159.stdout:982 write 0xc3b6e6f thru 0xc3c3c26 (0xcdb8 bytes) 2024-09-17T22:57:38.432 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:38.430+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:38.433 INFO:teuthology.orchestra.run.smithi159.stdout:983 read 0x3f8861a thru 0x3f8e0ee (0x5ad5 bytes) 2024-09-17T22:57:38.433 INFO:teuthology.orchestra.run.smithi159.stdout:984 write 0x8b6945 thru 0x8b9000 (0x26bc bytes) 2024-09-17T22:57:38.459 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.459 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:38.466 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:38.467 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:38.469 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:38.469 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:38.470 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:38.470 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T22:57:38.554 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:38.552+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:38.583 INFO:teuthology.orchestra.run.smithi159.stdout:985 punch from 0x9fae037 to 0x9fbbc2c, (0xdbf5 bytes) 2024-09-17T22:57:38.584 INFO:teuthology.orchestra.run.smithi159.stdout:986 read 0xaedf584 thru 0xaeecd94 (0xd811 bytes) 2024-09-17T22:57:38.584 INFO:teuthology.orchestra.run.smithi159.stdout:987 write 0x8c55b44 thru 0x8c5efef (0x94ac bytes) 2024-09-17T22:57:38.587 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.587 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.591 INFO:teuthology.orchestra.run.smithi159.stdout:988 write 0x943526 thru 0x944819 (0x12f4 bytes) 2024-09-17T22:57:38.598 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:38.608 INFO:teuthology.orchestra.run.smithi159.stdout:989 read 0xc2809fd thru 0xc286307 (0x590b bytes) 2024-09-17T22:57:38.618 INFO:teuthology.orchestra.run.smithi159.stdout:990 trunc from 0xce585b1 to 0x425d77d 2024-09-17T22:57:38.633 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:38.631+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:38.660 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:38.661 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:38.661 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T22:57:38.712 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.712 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.720 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:38.743 INFO:teuthology.orchestra.run.smithi159.stdout:991 write 0x918b9f3 thru 0x918c74f (0xd5d bytes) 2024-09-17T22:57:38.747 INFO:teuthology.orchestra.run.smithi159.stdout:994 write 0x863ad50 thru 0x864779e (0xca4f bytes) 2024-09-17T22:57:38.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:38.750 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.756 INFO:teuthology.orchestra.run.smithi159.stdout:996 writesame 0xa3f5db5 thru 0xa3f8bf1 data_size 0x13(0x2e3d bytes) 2024-09-17T22:57:38.757 INFO:teuthology.orchestra.run.smithi159.stdout:999 read 0x3ccc3f6 thru 0x3cd6e2c (0xaa37 bytes) 2024-09-17T22:57:38.758 INFO:teuthology.orchestra.run.smithi159.stdout:1000 write 0x2f330ba thru 0x2f3439b (0x12e2 bytes) 2024-09-17T22:57:38.759 INFO:teuthology.orchestra.run.smithi159.stdout:1001 read 0x65e40a3 thru 0x65edf7a (0x9ed8 bytes) 2024-09-17T22:57:38.760 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.760 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.760 INFO:teuthology.orchestra.run.smithi159.stdout:1005 read 0x1eef6ae thru 0x1ef7661 (0x7fb4 bytes) 2024-09-17T22:57:38.760 INFO:teuthology.orchestra.run.smithi159.stdout:1006 punch from 0xc867c5 to 0xc91285, (0xaac0 bytes) 2024-09-17T22:57:38.765 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:38.762+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:38.769 INFO:teuthology.orchestra.run.smithi159.stdout:1008 write 0xd2c1d4e thru 0xd2cb078 (0x932b bytes) 2024-09-17T22:57:38.773 INFO:teuthology.orchestra.run.smithi159.stdout:1012 write 0x43e1a96 thru 0x43e5882 (0x3ded bytes) 2024-09-17T22:57:38.774 INFO:teuthology.orchestra.run.smithi159.stdout:1014 read 0x681ba02 thru 0x6826a96 (0xb095 bytes) 2024-09-17T22:57:38.775 INFO:teuthology.orchestra.run.smithi159.stdout:1017 read 0xcbc1a83 thru 0xcbc8cc4 (0x7242 bytes) 2024-09-17T22:57:38.775 INFO:teuthology.orchestra.run.smithi159.stdout:1018 punch from 0x7569535 to 0x756c34e, (0x2e19 bytes) 2024-09-17T22:57:38.787 INFO:teuthology.orchestra.run.smithi159.stdout:1019 trunc from 0xd2cb079 to 0x38f9db2 2024-09-17T22:57:38.817 INFO:teuthology.orchestra.run.smithi159.stdout:1020 write 0xb600ec3 thru 0xb609672 (0x87b0 bytes) 2024-09-17T22:57:38.820 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.824 INFO:teuthology.orchestra.run.smithi159.stdout:1022 trunc from 0xb609673 to 0xe42e55c 2024-09-17T22:57:38.828 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.829 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.829 INFO:teuthology.orchestra.run.smithi159.stdout:1027 writesame 0x113e4f5 thru 0x1149679 data_size 0x5(0xb185 bytes) 2024-09-17T22:57:38.830 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.830 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:38.833 INFO:teuthology.orchestra.run.smithi159.stdout:1030 write 0x1afd196 thru 0x1b013b7 (0x4222 bytes) 2024-09-17T22:57:38.835 INFO:teuthology.orchestra.run.smithi159.stdout:1031 read 0x86d6030 thru 0x86d61dc (0x1ad bytes) 2024-09-17T22:57:38.835 INFO:teuthology.orchestra.run.smithi159.stdout:1032 read 0x3f71fbd thru 0x3f7ce3f (0xae83 bytes) 2024-09-17T22:57:38.835 INFO:teuthology.orchestra.run.smithi159.stdout:1034 read 0x3fb21c2 thru 0x3fb6742 (0x4581 bytes) 2024-09-17T22:57:38.836 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:38.832+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:38.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:38.841 INFO:teuthology.orchestra.run.smithi159.stdout:1037 write 0xe43f2cc thru 0xe44b684 (0xc3b9 bytes) 2024-09-17T22:57:38.847 INFO:teuthology.orchestra.run.smithi159.stdout:1038 write 0xe58a8c9 thru 0xe596524 (0xbc5c bytes) 2024-09-17T22:57:38.849 INFO:teuthology.orchestra.run.smithi159.stdout:1039 write 0x30da573 thru 0x30e2d10 (0x879e bytes) 2024-09-17T22:57:38.852 INFO:teuthology.orchestra.run.smithi159.stdout:1040 punch from 0xd956bab to 0xd963234, (0xc689 bytes) 2024-09-17T22:57:38.854 INFO:teuthology.orchestra.run.smithi159.stdout:1041 trunc from 0xe596525 to 0xc784665 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:38.860 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:38.861 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:38.861 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:38.861 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:38.861 INFO:teuthology.orchestra.run.smithi159.stdout:1043 write 0x4e0c6fd thru 0x4e180af (0xb9b3 bytes) 2024-09-17T22:57:38.862 INFO:teuthology.orchestra.run.smithi159.stdout:1045 read 0xc00c1b5 thru 0xc011082 (0x4ece bytes) 2024-09-17T22:57:38.862 INFO:teuthology.orchestra.run.smithi159.stdout:1047 writesame 0x4d7503c thru 0x4d7505c data_size 0x1(0x21 bytes) 2024-09-17T22:57:38.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:38.863 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:38.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T22:57:38.864 INFO:teuthology.orchestra.run.smithi159.stdout:1048 read 0x1f1fc85 thru 0x1f20930 (0xcac bytes) 2024-09-17T22:57:38.871 INFO:teuthology.orchestra.run.smithi159.stdout:1049 write 0xe09944e thru 0xe0a0e9f (0x7a52 bytes) 2024-09-17T22:57:38.874 INFO:teuthology.orchestra.run.smithi159.stdout:1050 write 0x51e5199 thru 0x51eb193 (0x5ffb bytes) 2024-09-17T22:57:38.879 INFO:teuthology.orchestra.run.smithi159.stdout:1052 trunc from 0xe0a0ea0 to 0xbd9efb4 2024-09-17T22:57:38.885 INFO:teuthology.orchestra.run.smithi159.stdout:1053 read 0x15d3932 thru 0x15dab08 (0x71d7 bytes) 2024-09-17T22:57:38.886 INFO:teuthology.orchestra.run.smithi159.stdout:1054 write 0x48906aa thru 0x489ab03 (0xa45a bytes) 2024-09-17T22:57:38.894 INFO:teuthology.orchestra.run.smithi159.stdout:1055 write 0xcd27b39 thru 0xcd35628 (0xdaf0 bytes) 2024-09-17T22:57:38.896 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:38.896 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:38.897 INFO:teuthology.orchestra.run.smithi159.stdout:1056 writesame 0x6ac3e76 thru 0x6acf216 data_size 0x11(0xb3a1 bytes) 2024-09-17T22:57:38.900 INFO:teuthology.orchestra.run.smithi159.stdout:1059 punch from 0x7a87d8f to 0x7a97673, (0xf8e4 bytes) 2024-09-17T22:57:38.900 INFO:teuthology.orchestra.run.smithi159.stdout:1061 writesame 0x2fcbd75 thru 0x2fd4fb8 data_size 0x5c(0x9244 bytes) 2024-09-17T22:57:38.901 INFO:teuthology.orchestra.run.smithi159.stdout:1062 write 0xc36148d thru 0xc3670b9 (0x5c2d bytes) 2024-09-17T22:57:38.904 INFO:teuthology.orchestra.run.smithi159.stdout:1064 write 0xae6e62c thru 0xae7182e (0x3203 bytes) 2024-09-17T22:57:38.905 INFO:teuthology.orchestra.run.smithi159.stdout:1065 write 0xc115f8a thru 0xc12399f (0xda16 bytes) 2024-09-17T22:57:38.908 INFO:teuthology.orchestra.run.smithi159.stdout:1066 write 0x6e0ed97 thru 0x6e0f64a (0x8b4 bytes) 2024-09-17T22:57:38.909 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.909 INFO:teuthology.orchestra.run.smithi159.stdout:1068 read 0x5e4dd98 thru 0x5e5599b (0x7c04 bytes) 2024-09-17T22:57:38.910 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.910 INFO:teuthology.orchestra.run.smithi159.stdout:1070 read 0x150f6c thru 0x1596d7 (0x876c bytes) 2024-09-17T22:57:38.910 INFO:teuthology.orchestra.run.smithi159.stdout:1071 read 0xa85e9d5 thru 0xa86c949 (0xdf75 bytes) 2024-09-17T22:57:38.917 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:38.917 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:38.922 INFO:teuthology.orchestra.run.smithi159.stdout:1073 trunc from 0xcd35629 to 0x28b5769 2024-09-17T22:57:38.937 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:38.937 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:38.938 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:38.934+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:38.938 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.956 INFO:teuthology.orchestra.run.smithi159.stdout:1075 write 0xb1e18e0 thru 0xb1eb2b9 (0x99da bytes) 2024-09-17T22:57:38.960 INFO:teuthology.orchestra.run.smithi159.stdout:1077 write 0x96da98c thru 0x96e5c9c (0xb311 bytes) 2024-09-17T22:57:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:38.962 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:1080 write 0x99b8599 thru 0x99c0c9e (0x8706 bytes) 2024-09-17T22:57:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:1081 read 0x58821af thru 0x588522d (0x307f bytes) 2024-09-17T22:57:38.966 INFO:teuthology.orchestra.run.smithi159.stdout:1082 writesame 0x3ad2b8d thru 0x3adacca data_size 0x47(0x813e bytes) 2024-09-17T22:57:38.967 INFO:teuthology.orchestra.run.smithi159.stdout:1083 punch from 0x6f174df to 0x6f1e9ae, (0x74cf bytes) 2024-09-17T22:57:38.967 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:38.979 INFO:teuthology.orchestra.run.smithi159.stdout:1086 trunc from 0xb1eb2ba to 0x16fa8c6 2024-09-17T22:57:38.980 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:38.980 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:38.993 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:38.993 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:39.001 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:39.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:39.004 INFO:teuthology.orchestra.run.smithi159.stdout:1087 write 0x636cb04 thru 0x637783e (0xad3b bytes) 2024-09-17T22:57:39.007 INFO:teuthology.orchestra.run.smithi159.stdout:1088 trunc from 0x637783f to 0x572c773 2024-09-17T22:57:39.024 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:39.024 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:39.024 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:39.025 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:39.026 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:39.026 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:39.026 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:39.026 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:39.027 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:39.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:39.028 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:39.029 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-17T22:57:39.030 INFO:teuthology.orchestra.run.smithi159.stdout:1089 write 0xd23af87 thru 0xd23e34e (0x33c8 bytes) 2024-09-17T22:57:39.032 INFO:teuthology.orchestra.run.smithi159.stdout:1090 write 0x55706b9 thru 0x557370c (0x3054 bytes) 2024-09-17T22:57:39.039 INFO:teuthology.orchestra.run.smithi159.stdout:1091 trunc from 0xd23e34f to 0x92ee7e9 2024-09-17T22:57:39.040 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:39.036+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:39.047 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:39.047 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:39.048 INFO:teuthology.orchestra.run.smithi159.stdout:1092 read 0x69e3639 thru 0x69ecf71 (0x9939 bytes) 2024-09-17T22:57:39.051 INFO:teuthology.orchestra.run.smithi159.stdout:1093 trunc from 0x92ee7e9 to 0xad53f10 2024-09-17T22:57:39.052 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:57:39.056 INFO:teuthology.orchestra.run.smithi159.stdout:1094 read 0x6155e8a thru 0x6160b30 (0xaca7 bytes) 2024-09-17T22:57:39.057 INFO:teuthology.orchestra.run.smithi159.stdout:1095 punch from 0x14da9ea to 0x14dc2a2, (0x18b8 bytes) 2024-09-17T22:57:39.057 INFO:teuthology.orchestra.run.smithi159.stdout:1096 read 0xd10c3e thru 0xd13847 (0x2c0a bytes) 2024-09-17T22:57:39.059 INFO:teuthology.orchestra.run.smithi159.stdout:1097 trunc from 0xad53f10 to 0xc2f443e 2024-09-17T22:57:39.066 INFO:teuthology.orchestra.run.smithi159.stdout:1098 trunc from 0xc2f443e to 0xa2cbf2d 2024-09-17T22:57:39.073 INFO:teuthology.orchestra.run.smithi159.stdout:1099 read 0x2043490 thru 0x2048088 (0x4bf9 bytes) 2024-09-17T22:57:39.073 INFO:teuthology.orchestra.run.smithi159.stdout:1100 write 0x9baed7c thru 0x9bbacf4 (0xbf79 bytes) 2024-09-17T22:57:39.087 INFO:teuthology.orchestra.run.smithi159.stdout:1101 trunc from 0xa2cbf2d to 0x28c36ac 2024-09-17T22:57:39.113 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:39.113 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:39.113 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:39.115 INFO:teuthology.orchestra.run.smithi159.stdout:1103 trunc from 0x28c36ac to 0xde64ff0 2024-09-17T22:57:39.121 INFO:teuthology.orchestra.run.smithi159.stdout:1104 read 0x4b51294 thru 0x4b5bfef (0xad5c bytes) 2024-09-17T22:57:39.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:39.134 INFO:teuthology.orchestra.run.smithi159.stdout:1105 trunc from 0xde64ff0 to 0x35d95bb 2024-09-17T22:57:39.138 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:39.136+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:39.147 INFO:teuthology.orchestra.run.smithi159.stdout:1107 writesame 0x422b9ab thru 0x422f439 data_size 0x13(0x3a8f bytes) 2024-09-17T22:57:39.148 INFO:teuthology.orchestra.run.smithi159.stdout:1108 read 0x108c6cd thru 0x1095ca5 (0x95d9 bytes) 2024-09-17T22:57:39.149 INFO:teuthology.orchestra.run.smithi159.stdout:1110 read 0x32aee1 thru 0x333544 (0x8664 bytes) 2024-09-17T22:57:39.155 INFO:teuthology.orchestra.run.smithi159.stdout:1111 writesame 0x443d8dd thru 0x4449ca5 data_size 0x3(0xc3c9 bytes) 2024-09-17T22:57:39.156 INFO:teuthology.orchestra.run.smithi159.stdout:1112 read 0x2d228cd thru 0x2d2c57e (0x9cb2 bytes) 2024-09-17T22:57:39.156 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.156 INFO:teuthology.orchestra.run.smithi159.stdout:1115 punch from 0x374628a to 0x374b742, (0x54b8 bytes) 2024-09-17T22:57:39.156 INFO:teuthology.orchestra.run.smithi159.stdout:1116 read 0x2608a66 thru 0x2611e90 (0x942b bytes) 2024-09-17T22:57:39.157 INFO:teuthology.orchestra.run.smithi159.stdout:1117 write 0x32fee39 thru 0x330c1d8 (0xd3a0 bytes) 2024-09-17T22:57:39.176 INFO:teuthology.orchestra.run.smithi159.stdout:1119 write 0xda5644c thru 0xda649ec (0xe5a1 bytes) 2024-09-17T22:57:39.179 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.179 INFO:teuthology.orchestra.run.smithi159.stdout:1123 writesame 0xd42a21d thru 0xd42a3d7 data_size 0x1(0x1bb bytes) 2024-09-17T22:57:39.182 INFO:teuthology.orchestra.run.smithi159.stdout:1124 read 0x2ff872a thru 0x30008de (0x81b5 bytes) 2024-09-17T22:57:39.182 INFO:teuthology.orchestra.run.smithi159.stdout:1125 read 0x89519b0 thru 0x8957b69 (0x61ba bytes) 2024-09-17T22:57:39.182 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.183 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.183 INFO:teuthology.orchestra.run.smithi159.stdout:1131 write 0x766bc47 thru 0x7678c0d (0xcfc7 bytes) 2024-09-17T22:57:39.184 INFO:teuthology.orchestra.run.smithi159.stdout:1132 read 0xac3044d thru 0xac37d07 (0x78bb bytes) 2024-09-17T22:57:39.184 INFO:teuthology.orchestra.run.smithi159.stdout:1134 writesame 0xcb6fdaa thru 0xcb7cf9f data_size 0x56(0xd1f6 bytes) 2024-09-17T22:57:39.189 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:39.189 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:39.189 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:39.190 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:39.193 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:39.193 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:39.193 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-17T22:57:39.194 INFO:teuthology.orchestra.run.smithi159.stdout:1135 writesame 0xe234d8c thru 0xe2441e6 data_size 0x1(0xf45b bytes) 2024-09-17T22:57:39.194 INFO:teuthology.orchestra.run.smithi159.stdout:1136 read 0x514747 thru 0x5184b2 (0x3d6c bytes) 2024-09-17T22:57:39.195 INFO:teuthology.orchestra.run.smithi159.stdout:1137 read 0x1346cb3 thru 0x1349f6a (0x32b8 bytes) 2024-09-17T22:57:39.195 INFO:teuthology.orchestra.run.smithi159.stdout:1138 read 0xdef8ebd thru 0xdf01f2f (0x9073 bytes) 2024-09-17T22:57:39.196 INFO:teuthology.orchestra.run.smithi159.stdout:1140 punch from 0x6c53378 to 0x6c559e3, (0x266b bytes) 2024-09-17T22:57:39.196 INFO:teuthology.orchestra.run.smithi159.stdout:1141 write 0x9c7a26d thru 0x9c803ec (0x6180 bytes) 2024-09-17T22:57:39.198 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.198 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.205 INFO:teuthology.orchestra.run.smithi159.stdout:1144 write 0xeae33e0 thru 0xeaeead0 (0xb6f1 bytes) 2024-09-17T22:57:39.206 INFO:teuthology.orchestra.run.smithi159.stdout:1145 punch from 0xd3f714e to 0xd40339d, (0xc24f bytes) 2024-09-17T22:57:39.222 INFO:teuthology.orchestra.run.smithi159.stdout:1147 trunc from 0xeaeead1 to 0x219e38a 2024-09-17T22:57:39.237 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:39.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 35.373900855999999, 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:39.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:57:39.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:57:39.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:39.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1472175159999995, 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:57:39.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1471829800000002, 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:57:39.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.253 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.147128597, 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:57:39.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:57:39.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1471035930000006, 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.258 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1470830119999995, 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:57:39.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1470617290000007, 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.267 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:39.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1470432030000008, 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.272 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1470250310000001, 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:57:39.277 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:57:39.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:39.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1470072029999994, 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:39.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:57:39.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:57:39.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1466920700000003, 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.288 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:57:39.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:57:39.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1459697989999995, 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:57:39.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:57:39.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:39.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1459252030000009, 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1458978809999998, 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:57:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:39.308 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1440790829999994, 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:57:39.312 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1437362780000004, 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.313 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:39.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1414651459999998, 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.141433052, 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:57:39.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1414065139999998, 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:57:39.327 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:57:39.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:57:39.331 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1413876209999998, 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:39.332 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:39.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:57:39.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:39.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:39.336 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:39.338 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1148 write 0x875bb99 thru 0x875f736 (0x3b9e bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1149 write 0xa580d55 thru 0xa58d4c4 (0xc770 bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1150 punch from 0x74ecafe to 0x74f6bb6, (0xa0b8 bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1151 write 0x4fccbec thru 0x4fdbf58 (0xf36d bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1154 read 0x368d5fd thru 0x3696dc7 (0x97cb bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1155 punch from 0x3947086 to 0x3949f7a, (0x2ef4 bytes) 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1158 trunc from 0xa58d4c5 to 0x78af312 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.340 INFO:teuthology.orchestra.run.smithi159.stdout:1160 writesame 0x9d291de thru 0x9d354b2 data_size 0x1(0xc2d5 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1161 write 0x387e179 thru 0x3886a4e (0x88d6 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1162 write 0x8d4dc6f thru 0x8d4ee04 (0x1196 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1163 punch from 0x31ce445 to 0x31d414e, (0x5d09 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1164 punch from 0x84a5df6 to 0x84a5e0f, (0x19 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1165 write 0xe7a6b7c thru 0xe7ade3b (0x72c0 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1166 write 0xa9a271a thru 0xa9ab02c (0x8913 bytes) 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1170 trunc from 0xe7ade3c to 0x696dc63 2024-09-17T22:57:39.341 INFO:teuthology.orchestra.run.smithi159.stdout:1171 write 0xbda3cb1 thru 0xbda6950 (0x2ca0 bytes) 2024-09-17T22:57:39.341 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:39.335+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:39.342 INFO:teuthology.orchestra.run.smithi159.stdout:1172 read 0x82facd5 thru 0x83068ea (0xbc16 bytes) 2024-09-17T22:57:39.345 INFO:teuthology.orchestra.run.smithi159.stdout:1173 write 0xc3b187d thru 0xc3bef99 (0xd71d bytes) 2024-09-17T22:57:39.346 INFO:teuthology.orchestra.run.smithi159.stdout:1176 write 0x9e7d11a thru 0x9e87bb3 (0xaa9a bytes) 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:57:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:bench [] [] [] [] 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:Error EINVAL: invalid command 2024-09-17T22:57:39.351 INFO:teuthology.orchestra.run.smithi159.stdout:1180 punch from 0xb2305d1 to 0xb23c522, (0xbf51 bytes) 2024-09-17T22:57:39.355 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:57:39.356 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:57:39.358 INFO:teuthology.orchestra.run.smithi159.stdout:1181 trunc from 0xc3bef9a to 0x5bcd653 2024-09-17T22:57:39.377 INFO:teuthology.orchestra.run.smithi159.stdout:1182 write 0x7b7e65b thru 0x7b7eb7f (0x525 bytes) 2024-09-17T22:57:39.378 INFO:teuthology.orchestra.run.smithi159.stdout:1183 writesame 0x68b059d thru 0x68b95e5 data_size 0x1(0x9049 bytes) 2024-09-17T22:57:39.394 INFO:teuthology.orchestra.run.smithi159.stdout:1184 read 0x5c8f58d thru 0x5c9c943 (0xd3b7 bytes) 2024-09-17T22:57:39.395 INFO:teuthology.orchestra.run.smithi159.stdout:1185 read 0x42e8c59 thru 0x42eaed0 (0x2278 bytes) 2024-09-17T22:57:39.409 INFO:teuthology.orchestra.run.smithi159.stdout:1186 writesame 0xd657582 thru 0xd65d32d data_size 0x2(0x5dac bytes) 2024-09-17T22:57:39.409 INFO:teuthology.orchestra.run.smithi159.stdout:1187 punch from 0x7ecd687 to 0x7ed8228, (0xaba1 bytes) 2024-09-17T22:57:39.409 INFO:teuthology.orchestra.run.smithi159.stdout:1188 write 0x281bdd1 thru 0x2821545 (0x5775 bytes) 2024-09-17T22:57:39.410 INFO:teuthology.orchestra.run.smithi159.stdout:1189 punch from 0xb74c6db to 0xb759e2b, (0xd750 bytes) 2024-09-17T22:57:39.410 INFO:teuthology.orchestra.run.smithi159.stdout:1191 read 0x90aa929 thru 0x90b3a36 (0x910e bytes) 2024-09-17T22:57:39.411 INFO:teuthology.orchestra.run.smithi159.stdout:1192 punch from 0xc803164 to 0xc8108cf, (0xd76b bytes) 2024-09-17T22:57:39.411 INFO:teuthology.orchestra.run.smithi159.stdout:1193 read 0x720d3a7 thru 0x7211859 (0x44b3 bytes) 2024-09-17T22:57:39.411 INFO:teuthology.orchestra.run.smithi159.stdout:1194 read 0xa9661e4 thru 0xa974658 (0xe475 bytes) 2024-09-17T22:57:39.411 INFO:teuthology.orchestra.run.smithi159.stdout:1195 write 0x7644dd2 thru 0x76487c0 (0x39ef bytes) 2024-09-17T22:57:39.412 INFO:teuthology.orchestra.run.smithi159.stdout:1196 write 0xc1f9fab thru 0xc1fe616 (0x466c bytes) 2024-09-17T22:57:39.421 INFO:teuthology.orchestra.run.smithi159.stdout:1198 trunc from 0xd65d32e to 0x95463a6 2024-09-17T22:57:39.427 INFO:teuthology.orchestra.run.smithi159.stdout:1199 write 0x7bae8af thru 0x7bbd7e3 (0xef35 bytes) 2024-09-17T22:57:39.437 INFO:teuthology.orchestra.run.smithi159.stdout:1200 trunc from 0x95463a6 to 0x3d23515 2024-09-17T22:57:39.438 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:39.435+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:39.469 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:39.469 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:39.469 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:39.470 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:39.478 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:39.562 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:39.560+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:39.621 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:39.622 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:39.622 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:39.622 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:39.622 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:39.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:39.650 INFO:teuthology.orchestra.run.smithi159.stdout:1205 read 0xf4b8fe thru 0xf4b974 (0x77 bytes) 2024-09-17T22:57:39.650 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:39.647+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:39.652 INFO:teuthology.orchestra.run.smithi159.stdout:1206 trunc from 0x3d23515 to 0x2b8e9fe 2024-09-17T22:57:39.668 INFO:teuthology.orchestra.run.smithi159.stdout:1207 writesame 0x5f07253 thru 0x5f0d43e data_size 0x6(0x61ec bytes) 2024-09-17T22:57:39.683 INFO:teuthology.orchestra.run.smithi159.stdout:1208 write 0xbb44b52 thru 0xbb48da4 (0x4253 bytes) 2024-09-17T22:57:39.687 INFO:teuthology.orchestra.run.smithi159.stdout:1209 read 0x5d3593a thru 0x5d35d10 (0x3d7 bytes) 2024-09-17T22:57:39.687 INFO:teuthology.orchestra.run.smithi159.stdout:1210 write 0x6f1b535 thru 0x6f237bb (0x8287 bytes) 2024-09-17T22:57:39.695 INFO:teuthology.orchestra.run.smithi159.stdout:1211 writesame 0xcaa8d2c thru 0xcaaf864 data_size 0x1(0x6b39 bytes) 2024-09-17T22:57:39.695 INFO:teuthology.orchestra.run.smithi159.stdout:1212 writesame 0xbd59898 thru 0xbd5c386 data_size 0x1(0x2aef bytes) 2024-09-17T22:57:39.700 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.712 INFO:teuthology.orchestra.run.smithi159.stdout:1214 trunc from 0xcaaf865 to 0x2ac7b3e 2024-09-17T22:57:39.732 INFO:teuthology.orchestra.run.smithi159.stdout:1215 trunc from 0x2ac7b3e to 0x9af7e6b 2024-09-17T22:57:39.737 INFO:teuthology.orchestra.run.smithi159.stdout:1216 read 0x79cd08d thru 0x79d7fdb (0xaf4f bytes) 2024-09-17T22:57:39.738 INFO:teuthology.orchestra.run.smithi159.stdout:1219 punch from 0x675c16 to 0x6775dd, (0x19c7 bytes) 2024-09-17T22:57:39.738 INFO:teuthology.orchestra.run.smithi159.stdout:1220 read 0x16b79e1 thru 0x16c3516 (0xbb36 bytes) 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:1221 punch from 0x1f383ce to 0x1f47d93, (0xf9c5 bytes) 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:1222 punch from 0x5192db5 to 0x51a2246, (0xf491 bytes) 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:1223 punch from 0x6b1d9c to 0x6bffcc, (0xe230 bytes) 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:1225 punch from 0x69e20a0 to 0x69ee2e9, (0xc249 bytes) 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.739 INFO:teuthology.orchestra.run.smithi159.stdout:1229 write 0x28c69b thru 0x28dd8f (0x16f5 bytes) 2024-09-17T22:57:39.740 INFO:teuthology.orchestra.run.smithi159.stdout:1230 read 0x4094ba6 thru 0x409ce0f (0x826a bytes) 2024-09-17T22:57:39.740 INFO:teuthology.orchestra.run.smithi159.stdout:1233 read 0x7c0389c thru 0x7c06eee (0x3653 bytes) 2024-09-17T22:57:39.740 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.741 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:39.738+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:39.746 INFO:teuthology.orchestra.run.smithi159.stdout:1235 trunc from 0x9af7e6b to 0xe75fa05 2024-09-17T22:57:39.751 INFO:teuthology.orchestra.run.smithi159.stdout:1236 punch from 0x75c4d41 to 0x75cf4aa, (0xa769 bytes) 2024-09-17T22:57:39.751 INFO:teuthology.orchestra.run.smithi159.stdout:1237 writesame 0xb800c9b thru 0xb810bd8 data_size 0x25(0xff3e bytes) 2024-09-17T22:57:39.771 INFO:teuthology.orchestra.run.smithi159.stdout:1239 trunc from 0xe75fa05 to 0x73945d 2024-09-17T22:57:39.773 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.505378372, 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:57:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16076 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5053608440000001, 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306234000000001, 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16076 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265290+0000", 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:57:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265300+0000", 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2520000000000005e-06 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265308+0000", 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2709999999999992e-06 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265313+0000", 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8770000000000001e-06 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318592+0000", 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053279435 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318596+0000", 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0389999999999998e-06 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5052743360000003, 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5052530750000002, 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:57:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5052317319999999, 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.505162275, 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:57:39.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.505143297, 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.801 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:39.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:57:39.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5051143680000001, 2024-09-17T22:57:39.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:57:39.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5050884299999998, 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:57:39.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:57:39.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5050604359999999, 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.814 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5050395119999997, 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:57:39.818 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:57:39.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.822 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.504992771, 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:57:39.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.504970363, 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:39.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5049470190000003, 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:57:39.831 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:57:39.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.835 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5049243300000001, 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:39.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:57:39.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5049021310000001, 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:57:39.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:57:39.843 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.504873345, 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.844 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:39.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:57:39.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5048507120000001, 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.849 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:57:39.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.853 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.504831459, 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.5048042829999999, 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.858 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:57:39.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:39.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.861 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:39.862 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:39.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:39.864 INFO:teuthology.orchestra.run.smithi159.stdout:1241 trunc from 0x73945d to 0xe70c685 2024-09-17T22:57:39.864 INFO:teuthology.orchestra.run.smithi159.stdout:1244 read 0x80aa842 thru 0x80b4c22 (0xa3e1 bytes) 2024-09-17T22:57:39.864 INFO:teuthology.orchestra.run.smithi159.stdout:1248 write 0x9f7b096 thru 0x9f84266 (0x91d1 bytes) 2024-09-17T22:57:39.864 INFO:teuthology.orchestra.run.smithi159.stdout:1249 writesame 0xc799783 thru 0xc7a1e75 data_size 0x1(0x86f3 bytes) 2024-09-17T22:57:39.865 INFO:teuthology.orchestra.run.smithi159.stdout:1250 punch from 0x75e314a to 0x75efdae, (0xcc64 bytes) 2024-09-17T22:57:39.865 INFO:teuthology.orchestra.run.smithi159.stdout:1251 trunc from 0xe70c685 to 0x8be89e2 2024-09-17T22:57:39.865 INFO:teuthology.orchestra.run.smithi159.stdout:1253 writesame 0xed174c6 thru 0xed19024 data_size 0x31(0x1b5f bytes) 2024-09-17T22:57:39.865 INFO:teuthology.orchestra.run.smithi159.stdout:1255 write 0x1fdf2c9 thru 0x1fe0a82 (0x17ba bytes) 2024-09-17T22:57:39.865 INFO:teuthology.orchestra.run.smithi159.stdout:1256 writesame 0xccab268 thru 0xccad636 data_size 0x1(0x23cf bytes) 2024-09-17T22:57:39.865 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:39.860+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:39.871 INFO:teuthology.orchestra.run.smithi159.stdout:1257 trunc from 0xed19025 to 0x62c24e3 2024-09-17T22:57:39.882 INFO:teuthology.orchestra.run.smithi159.stdout:1258 read 0x952bfc thru 0x961be2 (0xefe7 bytes) 2024-09-17T22:57:39.883 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.883 INFO:teuthology.orchestra.run.smithi159.stdout:1261 writesame 0x61c1c05 thru 0x61c56d3 data_size 0x1(0x3acf bytes) 2024-09-17T22:57:39.890 INFO:teuthology.orchestra.run.smithi159.stdout:1264 read 0x23d12cf thru 0x23dd53e (0xc270 bytes) 2024-09-17T22:57:39.891 INFO:teuthology.orchestra.run.smithi159.stdout:1265 writesame 0x154c0a8 thru 0x154ec37 data_size 0x8(0x2b90 bytes) 2024-09-17T22:57:39.892 INFO:teuthology.orchestra.run.smithi159.stdout:1267 read 0x31d5a4f thru 0x31dbd05 (0x62b7 bytes) 2024-09-17T22:57:39.901 INFO:teuthology.orchestra.run.smithi159.stdout:1268 write 0x8b6a4d1 thru 0x8b75cec (0xb81c bytes) 2024-09-17T22:57:39.904 INFO:teuthology.orchestra.run.smithi159.stdout:1270 read 0x6efadc1 thru 0x6f044c8 (0x9708 bytes) 2024-09-17T22:57:39.905 INFO:teuthology.orchestra.run.smithi159.stdout:1271 write 0x2b6aab3 thru 0x2b7656b (0xbab9 bytes) 2024-09-17T22:57:39.906 INFO:teuthology.orchestra.run.smithi159.stdout:1273 writesame 0x6fc22af thru 0x6fc8a7c data_size 0x6(0x67ce bytes) 2024-09-17T22:57:39.909 INFO:teuthology.orchestra.run.smithi159.stdout:1274 write 0x2dedf33 thru 0x2df4be6 (0x6cb4 bytes) 2024-09-17T22:57:39.916 INFO:teuthology.orchestra.run.smithi159.stdout:1275 trunc from 0x8b75ced to 0xd9325b3 2024-09-17T22:57:39.921 INFO:teuthology.orchestra.run.smithi159.stdout:1277 punch from 0x41bcd64 to 0x41ca318, (0xd5b4 bytes) 2024-09-17T22:57:39.921 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.927 INFO:teuthology.orchestra.run.smithi159.stdout:1282 write 0xe0fc3de thru 0xe107e05 (0xba28 bytes) 2024-09-17T22:57:39.930 INFO:teuthology.orchestra.run.smithi159.stdout:1283 punch from 0x19160d1 to 0x1919a56, (0x3985 bytes) 2024-09-17T22:57:39.936 INFO:teuthology.orchestra.run.smithi159.stdout:1284 write 0xebc67ab thru 0xebd355b (0xcdb1 bytes) 2024-09-17T22:57:39.956 INFO:teuthology.orchestra.run.smithi159.stdout:1285 trunc from 0xebd355c to 0x5c5324 2024-09-17T22:57:39.963 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:39.961+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:39.967 INFO:teuthology.orchestra.run.smithi159.stdout:1287 read 0x35bbee thru 0x35e834 (0x2c47 bytes) 2024-09-17T22:57:39.975 INFO:teuthology.orchestra.run.smithi159.stdout:1288 write 0x290aa76 thru 0x2918f4f (0xe4da bytes) 2024-09-17T22:57:39.977 INFO:teuthology.orchestra.run.smithi159.stdout:1289 write 0x8372cf thru 0x842e04 (0xbb36 bytes) 2024-09-17T22:57:39.991 INFO:teuthology.orchestra.run.smithi159.stdout:1290 writesame 0x9065e64 thru 0x9074134 data_size 0x69(0xe2d1 bytes) 2024-09-17T22:57:39.992 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:39.992 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:39.992 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:39.992 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:39.993 INFO:teuthology.orchestra.run.smithi159.stdout:1292 trunc from 0x9074135 to 0xaea4d52 2024-09-17T22:57:39.998 INFO:teuthology.orchestra.run.smithi159.stdout:1293 write 0xce0e25 thru 0xce1fe9 (0x11c5 bytes) 2024-09-17T22:57:39.999 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:39.999 INFO:teuthology.orchestra.run.smithi159.stdout:1295 write 0x4cdd754 thru 0x4cea611 (0xcebe bytes) 2024-09-17T22:57:40.000 INFO:teuthology.orchestra.run.smithi159.stdout:1296 write 0x872ce92 thru 0x873bcb0 (0xee1f bytes) 2024-09-17T22:57:40.002 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:40.004 INFO:teuthology.orchestra.run.smithi159.stdout:1297 trunc from 0xaea4d52 to 0xa9e21db 2024-09-17T22:57:40.009 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.009 INFO:teuthology.orchestra.run.smithi159.stdout:1299 writesame 0x5562a1c thru 0x556e5d1 data_size 0x3(0xbbb6 bytes) 2024-09-17T22:57:40.018 INFO:teuthology.orchestra.run.smithi159.stdout:1300 read 0x535a1fb thru 0x53661ed (0xbff3 bytes) 2024-09-17T22:57:40.018 INFO:teuthology.orchestra.run.smithi159.stdout:1301 punch from 0x782bccc to 0x782ff97, (0x42cb bytes) 2024-09-17T22:57:40.018 INFO:teuthology.orchestra.run.smithi159.stdout:1302 punch from 0x6842046 to 0x68474d7, (0x5491 bytes) 2024-09-17T22:57:40.021 INFO:teuthology.orchestra.run.smithi159.stdout:1303 trunc from 0xa9e21db to 0xcf87352 2024-09-17T22:57:40.032 INFO:teuthology.orchestra.run.smithi159.stdout:1304 trunc from 0xcf87352 to 0x7a447e8 2024-09-17T22:57:40.047 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:57:40.055 INFO:teuthology.orchestra.run.smithi159.stdout:1305 write 0x9d0f04f thru 0x9d1afa7 (0xbf59 bytes) 2024-09-17T22:57:40.063 INFO:teuthology.orchestra.run.smithi159.stdout:1307 writesame 0x5f6e03 thru 0x6064bd data_size 0x1(0xf6bb bytes) 2024-09-17T22:57:40.084 INFO:teuthology.orchestra.run.smithi159.stdout:1310 read 0x1b86b1c thru 0x1b88570 (0x1a55 bytes) 2024-09-17T22:57:40.084 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:40.081+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:40.084 INFO:teuthology.orchestra.run.smithi159.stdout:1311 write 0x2e08115 thru 0x2e0dc9d (0x5b89 bytes) 2024-09-17T22:57:40.085 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.085 INFO:teuthology.orchestra.run.smithi159.stdout:1314 write 0x5d7bdba thru 0x5d7f95b (0x3ba2 bytes) 2024-09-17T22:57:40.087 INFO:teuthology.orchestra.run.smithi159.stdout:1315 read 0x60d03ca thru 0x60d87b2 (0x83e9 bytes) 2024-09-17T22:57:40.087 INFO:teuthology.orchestra.run.smithi159.stdout:1317 write 0x3fd3d75 thru 0x3fdddb9 (0xa045 bytes) 2024-09-17T22:57:40.089 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.089 INFO:teuthology.orchestra.run.smithi159.stdout:1322 write 0x2095197 thru 0x20a2834 (0xd69e bytes) 2024-09-17T22:57:40.090 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.090 INFO:teuthology.orchestra.run.smithi159.stdout:1324 punch from 0x49e494e to 0x49e6226, (0x18d8 bytes) 2024-09-17T22:57:40.090 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.090 INFO:teuthology.orchestra.run.smithi159.stdout:1326 read 0x662be41 thru 0x6637190 (0xb350 bytes) 2024-09-17T22:57:40.100 INFO:teuthology.orchestra.run.smithi159.stdout:1327 write 0xd021771 thru 0xd0241d3 (0x2a63 bytes) 2024-09-17T22:57:40.109 INFO:teuthology.orchestra.run.smithi159.stdout:1328 write 0xe025448 thru 0xe02b49c (0x6055 bytes) 2024-09-17T22:57:40.111 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.118 INFO:teuthology.orchestra.run.smithi159.stdout:1330 trunc from 0xe02b49d to 0x8b87b49 2024-09-17T22:57:40.122 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:40.122 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:40.122 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:40.122 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:40.122 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:40.124 INFO:teuthology.orchestra.run.smithi159.stdout:1331 read 0x1c9760e thru 0x1ca395d (0xc350 bytes) 2024-09-17T22:57:40.130 INFO:teuthology.orchestra.run.smithi159.stdout:1333 write 0x9698b56 thru 0x96a8161 (0xf60c bytes) 2024-09-17T22:57:40.133 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:40.135 INFO:teuthology.orchestra.run.smithi159.stdout:1334 trunc from 0x96a8162 to 0x8dd438a 2024-09-17T22:57:40.140 INFO:teuthology.orchestra.run.smithi159.stdout:1335 write 0x8610e0a thru 0x8617c3e (0x6e35 bytes) 2024-09-17T22:57:40.141 INFO:teuthology.orchestra.run.smithi159.stdout:1336 read 0x8290167 thru 0x8291737 (0x15d1 bytes) 2024-09-17T22:57:40.142 INFO:teuthology.orchestra.run.smithi159.stdout:1338 writesame 0x2dfd341 thru 0x2e0712b data_size 0x1(0x9deb bytes) 2024-09-17T22:57:40.159 INFO:teuthology.orchestra.run.smithi159.stdout:1339 write 0x6a56d40 thru 0x6a636d1 (0xc992 bytes) 2024-09-17T22:57:40.160 INFO:teuthology.orchestra.run.smithi159.stdout:1340 punch from 0x8d6b5ea to 0x8d7a908, (0xf31e bytes) 2024-09-17T22:57:40.161 INFO:teuthology.orchestra.run.smithi159.stdout:1341 punch from 0x27bf353 to 0x27bf434, (0xe1 bytes) 2024-09-17T22:57:40.161 INFO:teuthology.orchestra.run.smithi159.stdout:1343 write 0x55766b6 thru 0x557b3f4 (0x4d3f bytes) 2024-09-17T22:57:40.162 INFO:teuthology.orchestra.run.smithi159.stdout:1344 punch from 0x8aa2c29 to 0x8aa340f, (0x7e6 bytes) 2024-09-17T22:57:40.162 INFO:teuthology.orchestra.run.smithi159.stdout:1345 writesame 0x1368d3f thru 0x1375122 data_size 0x7e(0xc3e4 bytes) 2024-09-17T22:57:40.163 INFO:teuthology.orchestra.run.smithi159.stdout:1346 read 0x1b87a23 thru 0x1b91789 (0x9d67 bytes) 2024-09-17T22:57:40.164 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:40.161+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:40.172 INFO:teuthology.orchestra.run.smithi159.stdout:1347 writesame 0xbd64877 thru 0xbd6dcca data_size 0x16(0x9454 bytes) 2024-09-17T22:57:40.177 INFO:teuthology.orchestra.run.smithi159.stdout:1348 trunc from 0xbd6dccb to 0x8fea342 2024-09-17T22:57:40.183 INFO:teuthology.orchestra.run.smithi159.stdout:1349 write 0x3c29f23 thru 0x3c30a82 (0x6b60 bytes) 2024-09-17T22:57:40.185 INFO:teuthology.orchestra.run.smithi159.stdout:1350 read 0x803f318 thru 0x8043600 (0x42e9 bytes) 2024-09-17T22:57:40.185 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.185 INFO:teuthology.orchestra.run.smithi159.stdout:1352 writesame 0x52f2cbc thru 0x5301dae data_size 0x1d(0xf0f3 bytes) 2024-09-17T22:57:40.187 INFO:teuthology.orchestra.run.smithi159.stdout:1353 read 0x2fa5390 thru 0x2fadd0e (0x897f bytes) 2024-09-17T22:57:40.188 INFO:teuthology.orchestra.run.smithi159.stdout:1357 read 0x7a14dcb thru 0x7a1eca1 (0x9ed7 bytes) 2024-09-17T22:57:40.193 INFO:teuthology.orchestra.run.smithi159.stdout:1358 trunc from 0x8fea342 to 0x5246ea5 2024-09-17T22:57:40.203 INFO:teuthology.orchestra.run.smithi159.stdout:1361 punch from 0x11fdc80 to 0x12032b6, (0x5636 bytes) 2024-09-17T22:57:40.203 INFO:teuthology.orchestra.run.smithi159.stdout:1362 punch from 0x366484 to 0x3706a3, (0xa21f bytes) 2024-09-17T22:57:40.204 INFO:teuthology.orchestra.run.smithi159.stdout:1363 trunc from 0x5246ea5 to 0x41bafff 2024-09-17T22:57:40.211 INFO:teuthology.orchestra.run.smithi159.stdout:1365 read 0x2a1db8a thru 0x2a297e4 (0xbc5b bytes) 2024-09-17T22:57:40.211 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.211 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.211 INFO:teuthology.orchestra.run.smithi159.stdout:1369 write 0x2cdb91 thru 0x2dc984 (0xedf4 bytes) 2024-09-17T22:57:40.222 INFO:teuthology.orchestra.run.smithi159.stdout:1370 trunc from 0x41bafff to 0xbf48e9b 2024-09-17T22:57:40.228 INFO:teuthology.orchestra.run.smithi159.stdout:1371 writesame 0x77139f5 thru 0x771acd9 data_size 0x1(0x72e5 bytes) 2024-09-17T22:57:40.242 INFO:teuthology.orchestra.run.smithi159.stdout:1372 read 0x98e6500 thru 0x98e7c39 (0x173a bytes) 2024-09-17T22:57:40.242 INFO:teuthology.orchestra.run.smithi159.stdout:1373 write 0x1468479 thru 0x14701c2 (0x7d4a bytes) 2024-09-17T22:57:40.243 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:40.244 INFO:teuthology.orchestra.run.smithi159.stdout:1377 clone 13 order 22 su 1048576 sc 4 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16134 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9959706810000002, 2024-09-17T22:57:40.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036277120000000003, 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16134 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255129+0000", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255131+0000", 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.916e-06 2024-09-17T22:57:40.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255137+0000", 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3509999999999998e-06 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255142+0000", 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.882e-06 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291253+0000", 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036111153 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291337+0000", 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3535000000000006e-05 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291405+0000", 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8858999999999999e-05 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16135 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9959507080000001, 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036429299999999998, 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16135 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255149+0000", 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:57:40.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255151+0000", 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255157+0000", 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255162+0000", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291437+0000", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036274726 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291514+0000", 2024-09-17T22:57:40.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7342000000000006e-05 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291578+0000", 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3634999999999996e-05 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16159 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9959309689999998, 2024-09-17T22:57:40.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036685328000000003, 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16159 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:40.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255170+0000", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255177+0000", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4989999999999999e-06 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255193+0000", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909999999999998e-05 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291665+0000", 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036471816999999997 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291751+0000", 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6509999999999997e-05 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291853+0000", 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102099 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16161 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9958934180000001, 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036865504, 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.268 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16161 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255208+0000", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3250000000000002e-06 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255218+0000", 2024-09-17T22:57:40.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4080000000000004e-06 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255223+0000", 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291884+0000", 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036661457000000001 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291966+0000", 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1920000000000002e-05 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292071+0000", 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000104608 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.272 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16163 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9958689679999999, 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037090170999999998, 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.273 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16163 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255233+0000", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255242+0000", 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0750000000000004e-06 2024-09-17T22:57:40.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255247+0000", 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292101+0000", 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036854162000000003 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292174+0000", 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2805000000000005e-05 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292320+0000", 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014646799999999999 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16165 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1728512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.277 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.99584553, 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037305086000000001, 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16165 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255254+0000", 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255256+0000", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.12e-06 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255265+0000", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8959999999999992e-06 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255270+0000", 2024-09-17T22:57:40.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8760000000000004e-06 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292365+0000", 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037095377999999998 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292452+0000", 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6947000000000006e-05 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292558+0000", 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010643 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16167 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.995822494, 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037507133999999998, 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16167 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255277+0000", 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255279+0000", 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255288+0000", 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8429999999999999e-06 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255293+0000", 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292603+0000", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037309869000000002 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292681+0000", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8484000000000003e-05 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292784+0000", 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010224 2024-09-17T22:57:40.286 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.287 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.288 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16169 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9957994729999999, 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037775437000000002, 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16169 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.289 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255300+0000", 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255302+0000", 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255308+0000", 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255313+0000", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292863+0000", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037549224999999999 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292969+0000", 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010604500000000001 2024-09-17T22:57:40.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293075+0000", 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010620699999999999 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16171 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1875968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9957795790000001, 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037958219000000001, 2024-09-17T22:57:40.293 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16171 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255320+0000", 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2899999999999999e-07 2024-09-17T22:57:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255322+0000", 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.906e-06 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255337+0000", 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5452999999999999e-05 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255343+0000", 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293106+0000", 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037763707000000001 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293186+0000", 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0153000000000003e-05 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293278+0000", 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1114999999999998e-05 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16172 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1925120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9957492960000001, 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038145196999999999, 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.298 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16172 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8400000000000004e-07 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255359+0000", 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:40.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255367+0000", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0649999999999999e-06 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255372+0000", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1390000000000003e-06 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293318+0000", 2024-09-17T22:57:40.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037945566 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293399+0000", 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1168999999999995e-05 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293495+0000", 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6162000000000005e-05 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.302 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16173 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.995718101, 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038380816999999998, 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.303 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16173 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255388+0000", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255402+0000", 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4532000000000001e-05 2024-09-17T22:57:40.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255408+0000", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7740000000000001e-06 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293577+0000", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038169526000000002 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293664+0000", 2024-09-17T22:57:40.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6312999999999998e-05 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293762+0000", 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7947000000000003e-05 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16174 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:40.307 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9956833449999998, 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038531986999999997, 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16174 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.308 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255418+0000", 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255426+0000", 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1620000000000002e-06 2024-09-17T22:57:40.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255436+0000", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0851e-05 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293804+0000", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038367215000000003 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293886+0000", 2024-09-17T22:57:40.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2189999999999997e-05 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293948+0000", 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1709000000000002e-05 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16176 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:40.312 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9956553919999998, 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038738529000000001, 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16176 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255446+0000", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255453+0000", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8170000000000001e-06 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255458+0000", 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:57:40.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293991+0000", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038532376 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294084+0000", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3614000000000002e-05 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294182+0000", 2024-09-17T22:57:40.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7949999999999996e-05 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16239 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9922829330000003, 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036338338999999997, 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.317 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16239 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258819+0000", 2024-09-17T22:57:40.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3e-06 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258825+0000", 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258830+0000", 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294968+0000", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036137423000000002 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295061+0000", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3177999999999995e-05 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295154+0000", 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3645000000000002e-05 2024-09-17T22:57:40.321 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16241 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1835008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9922583669999998, 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036566098999999998, 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.322 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16241 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4770000000000002e-06 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258850+0000", 2024-09-17T22:57:40.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5490000000000003e-06 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258855+0000", 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.904e-06 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295247+0000", 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036392378000000003 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295334+0000", 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7058999999999994e-05 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295407+0000", 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2287999999999996e-05 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16243 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9922365019999999, 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036705072999999998, 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16243 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:40.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258863+0000", 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258866+0000", 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.678e-06 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258872+0000", 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258877+0000", 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295445+0000", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036567582000000001 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295506+0000", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1895000000000003e-05 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295568+0000", 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1085000000000002e-05 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.330 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16245 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1933312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.991909916, 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036671423000000002, 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16245 2024-09-17T22:57:40.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259190+0000", 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259194+0000", 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259205+0000", 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0468e-05 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259212+0000", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1380000000000001e-06 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295666+0000", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036453700999999998 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295753+0000", 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7321e-05 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295860+0000", 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010761599999999999 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16246 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.9918753589999998, 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036882379, 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16246 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6599999999999996e-07 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259227+0000", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259234+0000", 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8539999999999999e-06 2024-09-17T22:57:40.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259239+0000", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295903+0000", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036664018999999999 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295993+0000", 2024-09-17T22:57:40.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9944000000000005e-05 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296106+0000", 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113158 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16248 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.339 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.991853189, 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037070651000000003, 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16248 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259248+0000", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259254+0000", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259260+0000", 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4489999999999998e-06 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296156+0000", 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036896089999999999 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296238+0000", 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2616999999999999e-05 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296316+0000", 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7964000000000001e-05 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16249 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.991682301, 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037100148999999999, 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.344 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16249 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8700000000000005e-07 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259421+0000", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0470000000000004e-06 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259431+0000", 2024-09-17T22:57:40.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4190000000000008e-06 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259437+0000", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4520000000000002e-06 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296352+0000", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036914354000000003 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296427+0000", 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5603000000000001e-05 2024-09-17T22:57:40.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296517+0000", 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9586999999999995e-05 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.348 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:40.350 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:40.350 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:40.346+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:40.450 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:40.448+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:40.493 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:40.493 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:40.494 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:40.494 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:40.502 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:40.551 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:40.548+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:40.645 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:40.645 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:40.645 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:40.645 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:40.645 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:40.652 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:40.649+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:40.654 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:40.752 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:40.750+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:40.802 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.908027020999999, 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1044 3.9 e28/22 3:954c0790:::rbd_data.118840f89003.0000000000000090:head v 28'4, mlcod=25'2)", 2024-09-17T22:57:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.770225317, 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032794199999999999, 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.09e-07 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028800+0000", 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7649999999999999e-05 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028812+0000", 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1865000000000001e-05 2024-09-17T22:57:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028819+0000", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7750000000000004e-06 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028870+0000", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0949000000000003e-05 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028889+0000", 2024-09-17T22:57:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9199999999999999e-05 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032019+0000", 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031302230000000001 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032061+0000", 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1949000000000001e-05 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 41 entries 28'5 (0'0) error 3:9656af7d:::rbd_data.118840f89003.0000000000000092:head by client.4488.0:1048 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.770152169999999, 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034247520000000001, 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028863+0000", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7249999999999997e-06 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028879+0000", 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5862000000000002e-05 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028892+0000", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3076e-05 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.029115+0000", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000223427 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032279+0000", 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.003164189 2024-09-17T22:57:40.816 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.4 epoch 28/22 rep_tid 35 entries 28'5 (0'0) error 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head by client.4488.0:1103 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'3)", 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.760443339, 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00299241, 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:40.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6199999999999997e-07 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038581+0000", 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7116e-05 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038603+0000", 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.177e-05 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038623+0000", 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9772e-05 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038712+0000", 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8910000000000001e-05 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.041556+0000", 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028440800000000001 2024-09-17T22:57:40.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 50 entries 28'7 (0'0) error 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head by client.4488.0:1110 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.759550044000001, 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0077516210000000002, 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039458+0000", 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.99e-07 2024-09-17T22:57:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039466+0000", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4419999999999998e-06 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039486+0000", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0523999999999999e-05 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039502+0000", 2024-09-17T22:57:40.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5930000000000002e-05 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039590+0000", 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8012999999999999e-05 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047208+0000", 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0076180129999999999 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.824 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1121 3.4 e28/22 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head v 28'6, mlcod=27'4)", 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.756447338999999, 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004866296, 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042566+0000", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042575+0000", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3349999999999996e-06 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042580+0000", 2024-09-17T22:57:40.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042623+0000", 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3396000000000003e-05 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042644+0000", 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1155999999999999e-05 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047379+0000", 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047352699999999998 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047426+0000", 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6424999999999997e-05 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.4 epoch 28/22 rep_tid 46 entries 28'7 (0'0) error 3:27450631:::rbd_data.118840f89003.00000000000000d0:head by client.4488.0:1269 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:40.829 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.088993+0000", 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.710013824000001, 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002145891, 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.088993+0000", 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.088993+0000", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.088994+0000", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.229e-06 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.089005+0000", 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.096e-05 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.089036+0000", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0521999999999997e-05 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.089054+0000", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8445999999999999e-05 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.089137+0000", 2024-09-17T22:57:40.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2871e-05 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.091139+0000", 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002001863 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:5169 3.9 e32/22 3:92748aee:::rbd_data.118846a2cbc0.0000000000000049:head v 32'12, mlcod=28'10)", 2024-09-17T22:57:40.833 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.226386955000001, 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029730630000000002, 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572621+0000", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.561e-06 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572630+0000", 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4020000000000002e-06 2024-09-17T22:57:40.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572649+0000", 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9216999999999999e-05 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572658+0000", 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5879999999999998e-06 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572723+0000", 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5339999999999994e-05 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572746+0000", 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3003e-05 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575556+0000", 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028103970000000001 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575593+0000", 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6554999999999998e-05 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:5175 3.11 e32/28 3:8af2d085:::rbd_data.118846a2cbc0.0000000000000070:head v 32'22, mlcod=32'20)", 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.573630+0000", 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.225376739, 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002192388, 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.838 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573630+0000", 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573630+0000", 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573631+0000", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9999999999999997e-07 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573640+0000", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7849999999999997e-06 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573653+0000", 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3664e-05 2024-09-17T22:57:40.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573660+0000", 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4949999999999996e-06 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573704+0000", 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.426e-05 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.573718+0000", 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4109e-05 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575778+0000", 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0020594770000000001 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575823+0000", 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4898000000000003e-05 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6483 3.9 e32/22 3:94ea7e0e:::rbd_data.118846a2cbc0.000000000000006b:head v 32'15, mlcod=32'13)", 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.804618+0000", 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.9943890119999992, 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023579410000000001, 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804618+0000", 2024-09-17T22:57:40.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804618+0000", 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804619+0000", 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0640000000000001e-06 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804636+0000", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6866000000000001e-05 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804649+0000", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3688e-05 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804657+0000", 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.96e-06 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804701+0000", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3881999999999998e-05 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.804717+0000", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6021000000000001e-05 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.806947+0000", 2024-09-17T22:57:40.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022300530000000001 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.806976+0000", 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8407000000000001e-05 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6635 3.4 e32/22 3:264a9651:::rbd_data.118846a2cbc0.0000000000000014:head v 32'17, mlcod=32'15)", 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.8658394749999996, 2024-09-17T22:57:40.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031534419999999998, 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933168+0000", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933175+0000", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8179999999999998e-06 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933185+0000", 2024-09-17T22:57:40.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0155e-05 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933191+0000", 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0499999999999997e-06 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933235+0000", 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3951e-05 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933250+0000", 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4605e-05 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936290+0000", 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030402879999999999 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936321+0000", 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0905999999999999e-05 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.851 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6750 3.4 e33/22 3:264a9651:::rbd_data.118846a2cbc0.0000000000000014:head v 33'21, mlcod=32'16)", 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.065131+0000", 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.7338759009999993, 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021274559999999998, 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065131+0000", 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065131+0000", 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065132+0000", 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0500000000000002e-07 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065138+0000", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2659999999999998e-06 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065151+0000", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2982e-05 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065159+0000", 2024-09-17T22:57:40.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7379999999999994e-06 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065206+0000", 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7160000000000002e-05 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.065226+0000", 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9643000000000001e-05 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.067225+0000", 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001999051 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.067258+0000", 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3810999999999999e-05 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6773 3.9 e33/22 3:906edfea:::rbd_data.118846a2cbc0.0000000000000044:head v 33'27, mlcod=33'23)", 2024-09-17T22:57:40.856 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.073722+0000", 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.7252852319999992, 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019873170000000002, 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073722+0000", 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073722+0000", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073722+0000", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4399999999999999e-07 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073741+0000", 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8059000000000001e-05 2024-09-17T22:57:40.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073753+0000", 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2863e-05 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073762+0000", 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5690000000000005e-06 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073818+0000", 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.571e-05 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.073840+0000", 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2362999999999999e-05 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075668+0000", 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018283080000000001 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075709+0000", 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0601e-05 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6774 3.0 e33/22 3:05c44946:::rbd_data.118846a2cbc0.0000000000000045:head v 33'36, mlcod=32'33)", 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.7235622920000004, 2024-09-17T22:57:40.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053547780000000001, 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:40.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0299999999999998e-07 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075451+0000", 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075468+0000", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7212999999999999e-05 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075474+0000", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075522+0000", 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8114e-05 2024-09-17T22:57:40.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075543+0000", 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0389000000000001e-05 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080753+0000", 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052103440000000004 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080799+0000", 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6335999999999998e-05 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.11 epoch 33/28 rep_tid 246 entries 33'38 (0'0) error 3:89c4a1e2:::rbd_header.118846a2cbc0:head by client.4488.0:7335 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'36)", 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.6040849159999997, 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023844890000000001, 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:40.866 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:40.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.06e-07 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194929+0000", 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1770000000000003e-06 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194948+0000", 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9021999999999998e-05 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194960+0000", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2248e-05 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195007+0000", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7064000000000002e-05 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.197306+0000", 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022994719999999999 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.869 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7337 3.11 e33/28 3:89c4a1e2:::rbd_header.118846a2cbc0:head v 33'39, mlcod=33'36)", 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.603718679, 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046186769999999999, 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195289+0000", 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195293+0000", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.155e-06 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195301+0000", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8520000000000004e-06 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195307+0000", 2024-09-17T22:57:40.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195340+0000", 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3090999999999999e-05 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195353+0000", 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2757e-05 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199876+0000", 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045229500000000004 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199907+0000", 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1340000000000001e-05 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14151 3.4 e36/22 3:2297f9f3:::rbd_data.11889f90e108.0000000000000068:head v 36'85, mlcod=36'83)", 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:34.734002+0000", 2024-09-17T22:57:40.874 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 6.0650052849999998, 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019783370000000002, 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734002+0000", 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734002+0000", 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734002+0000", 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8299999999999997e-07 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734007+0000", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8409999999999999e-06 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734021+0000", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3927999999999999e-05 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734027+0000", 2024-09-17T22:57:40.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734081+0000", 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4398000000000003e-05 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.734100+0000", 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9128999999999999e-05 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.735948+0000", 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001847418 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:34.735980+0000", 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2361999999999999e-05 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14579 3.11 e37/28 3:8ec16ab6:::rbd_data.11889f90e108.0000000000000148:head v 37'73, mlcod=36'71)", 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:40.879 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.7199738949999999, 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002650075, 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.880 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079035+0000", 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7239999999999999e-06 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079044+0000", 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3780000000000007e-06 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079063+0000", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9412999999999999e-05 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079074+0000", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0366999999999999e-05 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079135+0000", 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1362e-05 2024-09-17T22:57:40.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079158+0000", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2388e-05 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081639+0000", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002481797 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081683+0000", 2024-09-17T22:57:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3646000000000002e-05 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14587 3.4 e37/22 3:23430168:::rbd_data.11889f90e108.000000000000021d:head v 37'90, mlcod=37'88)", 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.7173478749999997, 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002639691, 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.884 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081660+0000", 2024-09-17T22:57:40.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6599999999999996e-07 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081666+0000", 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081675+0000", 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5279999999999992e-06 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.886 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081682+0000", 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7129999999999999e-06 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081730+0000", 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8174000000000003e-05 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081749+0000", 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9279999999999998e-05 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084252+0000", 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025027280000000001 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084299+0000", 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6601000000000003e-05 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.888 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14599 3.0 e37/22 3:03d9ef58:::rbd_data.11889f90e108.000000000000035d:head v 37'60, mlcod=37'58)", 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.7143676819999998, 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023997279999999998, 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.889 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084640+0000", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5799999999999998e-07 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084646+0000", 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2299999999999996e-06 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084657+0000", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1137e-05 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084664+0000", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084719+0000", 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5336e-05 2024-09-17T22:57:40.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084738+0000", 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9122000000000001e-05 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.086996+0000", 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022571259999999999 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:40.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.087039+0000", 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3290000000000001e-05 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:40.893 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:40.894 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:40.895 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:40.892+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:40.995 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:40.993+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:41.038 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:41.038 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:41.038 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:41.038 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:41.048 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:41.062 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:41.096 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:41.093+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:41.161 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:41.161 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:41.161 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:41.161 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:41.161 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:41.171 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:41.174 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone12 from 0xbf48e9b (overlap 0x5c5324) to 0x1bb28a 2024-09-17T22:57:41.197 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:41.194+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:41.223 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:41.224 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:41.242 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:41.242 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:41.259 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:41.260 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:41.277 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:41.277 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:41.289 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.289 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.294 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:41.295 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:41.298 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:41.295+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:41.300 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:41.312 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:41.313 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:41.330 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:41.330 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:41.332 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T22:57:41.347 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:41.347 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:41.398 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:41.395+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:41.412 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.412 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.421 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:41.505 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:41.502+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:41.565 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.565 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:41.589 INFO:teuthology.orchestra.run.smithi159.stdout:1378 read 0x733d61f thru 0x734c681 (0xf063 bytes) 2024-09-17T22:57:41.591 INFO:teuthology.orchestra.run.smithi159.stdout:1379 write 0x9387e1d thru 0x93936ed (0xb8d1 bytes) 2024-09-17T22:57:41.598 INFO:teuthology.orchestra.run.smithi159.stdout:1381 write 0xa338fc9 thru 0xa33b38b (0x23c3 bytes) 2024-09-17T22:57:41.613 INFO:teuthology.orchestra.run.smithi159.stdout:1382 trunc from 0xbf48e9b to 0x26c975b 2024-09-17T22:57:41.615 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:41.612+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:41.635 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.635 INFO:teuthology.orchestra.run.smithi159.stdout:1384 read 0x1086cc6 thru 0x10912d3 (0xa60e bytes) 2024-09-17T22:57:41.636 INFO:teuthology.orchestra.run.smithi159.stdout:1385 read 0xb09019 thru 0xb185f4 (0xf5dc bytes) 2024-09-17T22:57:41.636 INFO:teuthology.orchestra.run.smithi159.stdout:1386 read 0x16b1b9b thru 0x16b3b21 (0x1f87 bytes) 2024-09-17T22:57:41.637 INFO:teuthology.orchestra.run.smithi159.stdout:1387 punch from 0x208391e to 0x208b2a9, (0x798b bytes) 2024-09-17T22:57:41.637 INFO:teuthology.orchestra.run.smithi159.stdout:1388 read 0x101fe24 thru 0x102d4ee (0xd6cb bytes) 2024-09-17T22:57:41.639 INFO:teuthology.orchestra.run.smithi159.stdout:1389 trunc from 0x26c975b to 0x366ab8f 2024-09-17T22:57:41.644 INFO:teuthology.orchestra.run.smithi159.stdout:1390 punch from 0x2838f2b to 0x283b310, (0x23e5 bytes) 2024-09-17T22:57:41.644 INFO:teuthology.orchestra.run.smithi159.stdout:1391 punch from 0x3593eca to 0x359d9d9, (0x9b0f bytes) 2024-09-17T22:57:41.654 INFO:teuthology.orchestra.run.smithi159.stdout:1392 trunc from 0x366ab8f to 0xc61841f 2024-09-17T22:57:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:1393 trunc from 0xc61841f to 0xb47d1f7 2024-09-17T22:57:41.667 INFO:teuthology.orchestra.run.smithi159.stdout:1394 read 0xa5e388b thru 0xa5e7dc6 (0x453c bytes) 2024-09-17T22:57:41.667 INFO:teuthology.orchestra.run.smithi159.stdout:1395 punch from 0x4b91a48 to 0x4b9f49e, (0xda56 bytes) 2024-09-17T22:57:41.677 INFO:teuthology.orchestra.run.smithi159.stdout:1396 write 0xca2e5df thru 0xca38df9 (0xa81b bytes) 2024-09-17T22:57:41.694 INFO:teuthology.orchestra.run.smithi159.stdout:1397 trunc from 0xca38dfa to 0x7cfc79 2024-09-17T22:57:41.700 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:41.698+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:41.723 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.724 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.727 INFO:teuthology.orchestra.run.smithi159.stdout:1398 read 0x546b9a thru 0x5536b8 (0xcb1f bytes) 2024-09-17T22:57:41.728 INFO:teuthology.orchestra.run.smithi159.stdout:1399 read 0x762ae0 thru 0x76b4f3 (0x8a14 bytes) 2024-09-17T22:57:41.733 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:57:41.736 INFO:teuthology.orchestra.run.smithi159.stdout:1400 writesame 0x2a42ad0 thru 0x2a439a7 data_size 0x2(0xed8 bytes) 2024-09-17T22:57:41.737 INFO:teuthology.orchestra.run.smithi159.stdout:1401 read 0x7d5806 thru 0x7d7ba0 (0x239b bytes) 2024-09-17T22:57:41.737 INFO:teuthology.orchestra.run.smithi159.stdout:1402 writesame 0x1c91671 thru 0x1c9724a data_size 0x3(0x5bda bytes) 2024-09-17T22:57:41.743 INFO:teuthology.orchestra.run.smithi159.stdout:1403 write 0x191a701 thru 0x1921a2e (0x732e bytes) 2024-09-17T22:57:41.748 INFO:teuthology.orchestra.run.smithi159.stdout:1405 read 0xd681ec thru 0xd68f76 (0xd8b bytes) 2024-09-17T22:57:41.751 INFO:teuthology.orchestra.run.smithi159.stdout:1406 trunc from 0x2a439a8 to 0x853f8e 2024-09-17T22:57:41.783 INFO:teuthology.orchestra.run.smithi159.stdout:1407 write 0xa6a8972 thru 0xa6b45ed (0xbc7c bytes) 2024-09-17T22:57:41.786 INFO:teuthology.orchestra.run.smithi159.stdout:1408 read 0x17f2f0c thru 0x17faffd (0x80f2 bytes) 2024-09-17T22:57:41.786 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.786 INFO:teuthology.orchestra.run.smithi159.stdout:1411 read 0x8d7865e thru 0x8d87437 (0xedda bytes) 2024-09-17T22:57:41.786 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.786 INFO:teuthology.orchestra.run.smithi159.stdout:1414 write 0x688b7f9 thru 0x6895f44 (0xa74c bytes) 2024-09-17T22:57:41.789 INFO:teuthology.orchestra.run.smithi159.stdout:1415 read 0x7df3e69 thru 0x7e01bb3 (0xdd4b bytes) 2024-09-17T22:57:41.796 INFO:teuthology.orchestra.run.smithi159.stdout:1417 trunc from 0xa6b45ee to 0x4a56db0 2024-09-17T22:57:41.801 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:41.799+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:41.806 INFO:teuthology.orchestra.run.smithi159.stdout:1419 punch from 0xdd3347 to 0xde2fb3, (0xfc6c bytes) 2024-09-17T22:57:41.806 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.806 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.806 INFO:teuthology.orchestra.run.smithi159.stdout:1422 punch from 0x2a861b2 to 0x2a9193a, (0xb788 bytes) 2024-09-17T22:57:41.809 INFO:teuthology.orchestra.run.smithi159.stdout:1423 trunc from 0x4a56db0 to 0x1ef4519 2024-09-17T22:57:41.814 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.821 INFO:teuthology.orchestra.run.smithi159.stdout:1425 trunc from 0x1ef4519 to 0x7d3da77 2024-09-17T22:57:41.827 INFO:teuthology.orchestra.run.smithi159.stdout:1426 write 0x59b1d4f thru 0x59b4d8f (0x3041 bytes) 2024-09-17T22:57:41.829 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.829 INFO:teuthology.orchestra.run.smithi159.stdout:1428 read 0xabb29 thru 0xb807b (0xc553 bytes) 2024-09-17T22:57:41.830 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:41.830 INFO:teuthology.orchestra.run.smithi159.stdout:1430 read 0x3669886 thru 0x367179e (0x7f19 bytes) 2024-09-17T22:57:41.830 INFO:teuthology.orchestra.run.smithi159.stdout:1431 read 0x73f1284 thru 0x73ff0a3 (0xde20 bytes) 2024-09-17T22:57:41.830 INFO:teuthology.orchestra.run.smithi159.stdout:1432 clone 14 order 18 su 8192 sc 15 2024-09-17T22:57:41.840 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.840 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.849 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:57:41.902 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:41.900+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:41.953 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:41.953 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:41.962 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:57:42.003 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:42.000+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:42.064 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:57:42.068 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.069 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:42.107 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:42.105+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:42.171 INFO:teuthology.orchestra.run.smithi159.stdout:flattening image image_client.0-clone13 2024-09-17T22:57:42.187 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.187 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.197 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:42.205 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:42.203+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:42.305 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.306 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.306 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:42.303+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:42.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:42.328 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T22:57:42.335 INFO:teuthology.orchestra.run.smithi159.stdout:1433 punch from 0x5715fd3 to 0x5717ab0, (0x1add bytes) 2024-09-17T22:57:42.340 INFO:teuthology.orchestra.run.smithi159.stdout:1435 punch from 0x63a4592 to 0x63b2fd0, (0xea3e bytes) 2024-09-17T22:57:42.340 INFO:teuthology.orchestra.run.smithi159.stdout:1436 write 0x76e5aa8 thru 0x76e981b (0x3d74 bytes) 2024-09-17T22:57:42.344 INFO:teuthology.orchestra.run.smithi159.stdout:1437 write 0xf2309b thru 0xf29bca (0x6b30 bytes) 2024-09-17T22:57:42.366 INFO:teuthology.orchestra.run.smithi159.stdout:1438 writesame 0xc7c0f56 thru 0xc7c52f5 data_size 0x10(0x43a0 bytes) 2024-09-17T22:57:42.367 INFO:teuthology.orchestra.run.smithi159.stdout:1439 read 0xb15c7fe thru 0xb1684fe (0xbd01 bytes) 2024-09-17T22:57:42.367 INFO:teuthology.orchestra.run.smithi159.stdout:1440 write 0x31ee8f7 thru 0x31fe706 (0xfe10 bytes) 2024-09-17T22:57:42.378 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.378 INFO:teuthology.orchestra.run.smithi159.stdout:1442 read 0x31d7677 thru 0x31e11a6 (0x9b30 bytes) 2024-09-17T22:57:42.378 INFO:teuthology.orchestra.run.smithi159.stdout:1443 read 0xb8f8be8 thru 0xb9009b1 (0x7dca bytes) 2024-09-17T22:57:42.379 INFO:teuthology.orchestra.run.smithi159.stdout:1444 write 0x46414da thru 0x4648351 (0x6e78 bytes) 2024-09-17T22:57:42.390 INFO:teuthology.orchestra.run.smithi159.stdout:1445 punch from 0x634adf9 to 0x635719e, (0xc3a5 bytes) 2024-09-17T22:57:42.391 INFO:teuthology.orchestra.run.smithi159.stdout:1447 write 0x10ee350 thru 0x10ee733 (0x3e4 bytes) 2024-09-17T22:57:42.404 INFO:teuthology.orchestra.run.smithi159.stdout:1448 write 0xddf1cab thru 0xddfaf4c (0x92a2 bytes) 2024-09-17T22:57:42.407 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:42.404+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:42.414 INFO:teuthology.orchestra.run.smithi159.stdout:1449 punch from 0x6071e90 to 0x60774b4, (0x5624 bytes) 2024-09-17T22:57:42.414 INFO:teuthology.orchestra.run.smithi159.stdout:1450 write 0x8394bfb thru 0x839cbd1 (0x7fd7 bytes) 2024-09-17T22:57:42.424 INFO:teuthology.orchestra.run.smithi159.stdout:1453 writesame 0x3035cd2 thru 0x303bebd data_size 0x6(0x61ec bytes) 2024-09-17T22:57:42.435 INFO:teuthology.orchestra.run.smithi159.stdout:1454 trunc from 0xddfaf4d to 0x9f63fae 2024-09-17T22:57:42.445 INFO:teuthology.orchestra.run.smithi159.stdout:1455 writesame 0x702d21c thru 0x7030ad2 data_size 0x1(0x38b7 bytes) 2024-09-17T22:57:42.458 INFO:teuthology.orchestra.run.smithi159.stdout:1456 write 0xa77db20 thru 0xa77f281 (0x1762 bytes) 2024-09-17T22:57:42.462 INFO:teuthology.orchestra.run.smithi159.stdout:1458 write 0x64d0afb thru 0x64dd8b3 (0xcdb9 bytes) 2024-09-17T22:57:42.469 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.469 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.471 INFO:teuthology.orchestra.run.smithi159.stdout:1459 punch from 0x5f116c8 to 0x5f207a8, (0xf0e0 bytes) 2024-09-17T22:57:42.471 INFO:teuthology.orchestra.run.smithi159.stdout:1461 writesame 0x2b85fb7 thru 0x2b90b7f data_size 0x6b(0xabc9 bytes) 2024-09-17T22:57:42.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:42.480 INFO:teuthology.orchestra.run.smithi159.stdout:1462 write 0x76361f5 thru 0x763daea (0x78f6 bytes) 2024-09-17T22:57:42.495 INFO:teuthology.orchestra.run.smithi159.stdout:1464 trunc from 0xa77f282 to 0x4d51af9 2024-09-17T22:57:42.508 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:42.505+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:42.540 INFO:teuthology.orchestra.run.smithi159.stdout:1465 write 0xcb07f5d thru 0xcb093ab (0x144f bytes) 2024-09-17T22:57:42.543 INFO:teuthology.orchestra.run.smithi159.stdout:1466 write 0x6661c72 thru 0x666306e (0x13fd bytes) 2024-09-17T22:57:42.547 INFO:teuthology.orchestra.run.smithi159.stdout:1467 writesame 0x8bf9113 thru 0x8c043b6 data_size 0x25(0xb2a4 bytes) 2024-09-17T22:57:42.559 INFO:teuthology.orchestra.run.smithi159.stdout:1472 read 0x5a98036 thru 0x5a9a9ba (0x2985 bytes) 2024-09-17T22:57:42.565 INFO:teuthology.orchestra.run.smithi159.stdout:1473 writesame 0xdc1c25b thru 0xdc28863 data_size 0x2b(0xc609 bytes) 2024-09-17T22:57:42.565 INFO:teuthology.orchestra.run.smithi159.stdout:1474 read 0xa98405d thru 0xa984d1f (0xcc3 bytes) 2024-09-17T22:57:42.566 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.566 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.566 INFO:teuthology.orchestra.run.smithi159.stdout:1477 writesame 0x83ff4e7 thru 0x840b414 data_size 0x9(0xbf2e bytes) 2024-09-17T22:57:42.579 INFO:teuthology.orchestra.run.smithi159.stdout:1478 write 0x3b8ca1b thru 0x3b99c47 (0xd22d bytes) 2024-09-17T22:57:42.591 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.591 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.591 INFO:teuthology.orchestra.run.smithi159.stdout:1482 read 0x8f4766e thru 0x8f54186 (0xcb19 bytes) 2024-09-17T22:57:42.591 INFO:teuthology.orchestra.run.smithi159.stdout:1483 writesame 0x3a13bdc thru 0x3a19c91 data_size 0x1(0x60b6 bytes) 2024-09-17T22:57:42.602 INFO:teuthology.orchestra.run.smithi159.stdout:1484 write 0x2bf410c thru 0x2bffb63 (0xba58 bytes) 2024-09-17T22:57:42.608 INFO:teuthology.orchestra.run.smithi159.stdout:1486 write 0x34b2164 thru 0x34be3e0 (0xc27d bytes) 2024-09-17T22:57:42.608 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:42.605+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:42.616 INFO:teuthology.orchestra.run.smithi159.stdout:1488 writesame 0x3f9fc7 thru 0x4033bc data_size 0x3b(0x93f6 bytes) 2024-09-17T22:57:42.630 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.630 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.639 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:42.641 INFO:teuthology.orchestra.run.smithi159.stdout:1489 read 0xc7a9d62 thru 0xc7b75ba (0xd859 bytes) 2024-09-17T22:57:42.642 INFO:teuthology.orchestra.run.smithi159.stdout:1490 write 0x50ad0c7 thru 0x50b76e1 (0xa61b bytes) 2024-09-17T22:57:42.652 INFO:teuthology.orchestra.run.smithi159.stdout:1491 read 0x6990532 thru 0x6993ef5 (0x39c4 bytes) 2024-09-17T22:57:42.652 INFO:teuthology.orchestra.run.smithi159.stdout:truncating to largest ever: 0xec79bb8 2024-09-17T22:57:42.653 INFO:teuthology.orchestra.run.smithi159.stdout:1494 trunc from 0xdc28864 to 0xec79bb8 2024-09-17T22:57:42.658 INFO:teuthology.orchestra.run.smithi159.stdout:1496 read 0x833518 thru 0x836a33 (0x351c bytes) 2024-09-17T22:57:42.659 INFO:teuthology.orchestra.run.smithi159.stdout:1498 write 0xcbf02ee thru 0xcbf4cee (0x4a01 bytes) 2024-09-17T22:57:42.680 INFO:teuthology.orchestra.run.smithi159.stdout:1499 trunc from 0xec79bb8 to 0x4084eb6 2024-09-17T22:57:42.723 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:42.720+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:1501 write 0x85ca486 thru 0x85cf98b (0x5506 bytes) 2024-09-17T22:57:42.740 INFO:teuthology.orchestra.run.smithi159.stdout:1502 write 0x20853b thru 0x20df63 (0x5a29 bytes) 2024-09-17T22:57:42.752 INFO:teuthology.orchestra.run.smithi159.stdout:1503 read 0x2249d9a thru 0x225731a (0xd581 bytes) 2024-09-17T22:57:42.752 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:42.752 INFO:teuthology.orchestra.run.smithi159.stdout:1506 read 0xaf9ee0 thru 0xafa7cc (0x8ed bytes) 2024-09-17T22:57:42.752 INFO:teuthology.orchestra.run.smithi159.stdout:1507 read 0x428f186 thru 0x4294ff8 (0x5e73 bytes) 2024-09-17T22:57:42.753 INFO:teuthology.orchestra.run.smithi159.stdout:1508 writesame 0x1226a4b thru 0x1227020 data_size 0x9(0x5d6 bytes) 2024-09-17T22:57:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:1509 read 0x1874fea thru 0x1879075 (0x408c bytes) 2024-09-17T22:57:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:1511 read 0x3ee3763 thru 0x3ee8cfa (0x5598 bytes) 2024-09-17T22:57:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:1512 punch from 0x28e1d59 to 0x28ecdeb, (0xb092 bytes) 2024-09-17T22:57:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:1514 punch from 0x4850232 to 0x4851009, (0xdd7 bytes) 2024-09-17T22:57:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:1515 trunc from 0x85cf98c to 0x8569c1d 2024-09-17T22:57:42.764 INFO:teuthology.orchestra.run.smithi159.stdout:1516 write 0x30087aa thru 0x3015040 (0xc897 bytes) 2024-09-17T22:57:42.776 INFO:teuthology.orchestra.run.smithi159.stdout:1517 write 0x9b7a9e thru 0x9c6259 (0xe7bc bytes) 2024-09-17T22:57:42.795 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.795 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.796 INFO:teuthology.orchestra.run.smithi159.stdout:1518 punch from 0x20d785 to 0x2137b9, (0x6034 bytes) 2024-09-17T22:57:42.796 INFO:teuthology.orchestra.run.smithi159.stdout:1520 writesame 0x51c663c thru 0x51c6f3e data_size 0x1(0x903 bytes) 2024-09-17T22:57:42.799 INFO:teuthology.orchestra.run.smithi159.stdout:1521 writesame 0x275cfa3 thru 0x275e0bb data_size 0x1(0x1119 bytes) 2024-09-17T22:57:42.804 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:42.805 INFO:teuthology.orchestra.run.smithi159.stdout:1522 punch from 0x28de117 to 0x28e70c2, (0x8fab bytes) 2024-09-17T22:57:42.814 INFO:teuthology.orchestra.run.smithi159.stdout:1525 write 0xb7f3672 thru 0xb801952 (0xe2e1 bytes) 2024-09-17T22:57:42.823 INFO:teuthology.orchestra.run.smithi159.stdout:1526 writesame 0xb273062 thru 0xb27ca26 data_size 0x1(0x99c5 bytes) 2024-09-17T22:57:42.837 INFO:teuthology.orchestra.run.smithi159.stdout:1528 read 0xb17411b thru 0xb181a8e (0xd974 bytes) 2024-09-17T22:57:42.837 INFO:teuthology.orchestra.run.smithi159.stdout:1529 read 0x48b398b thru 0x48b930a (0x5980 bytes) 2024-09-17T22:57:42.837 INFO:teuthology.orchestra.run.smithi159.stdout:1531 write 0x4bc0446 thru 0x4bce3a4 (0xdf5f bytes) 2024-09-17T22:57:42.845 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:42.842+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:42.849 INFO:teuthology.orchestra.run.smithi159.stdout:1534 trunc from 0xb801953 to 0x93ddba8 2024-09-17T22:57:42.860 INFO:teuthology.orchestra.run.smithi159.stdout:1535 read 0x5dc420a thru 0x5dcec24 (0xaa1b bytes) 2024-09-17T22:57:42.871 INFO:teuthology.orchestra.run.smithi159.stdout:1536 writesame 0xc6a2990 thru 0xc6b138b data_size 0xa(0xe9fc bytes) 2024-09-17T22:57:42.871 INFO:teuthology.orchestra.run.smithi159.stdout:1537 writesame 0x22dd3df thru 0x22ebeb1 data_size 0xb(0xead3 bytes) 2024-09-17T22:57:42.880 INFO:teuthology.orchestra.run.smithi159.stdout:1538 read 0x29cc030 thru 0x29cd583 (0x1554 bytes) 2024-09-17T22:57:42.886 INFO:teuthology.orchestra.run.smithi159.stdout:1539 trunc from 0xc6b138c to 0x801f608 2024-09-17T22:57:42.895 INFO:teuthology.orchestra.run.smithi159.stdout:1541 trunc from 0x801f608 to 0x7e1cc79 2024-09-17T22:57:42.900 INFO:teuthology.orchestra.run.smithi159.stdout:1542 read 0xa9f769 thru 0xaa3f4a (0x47e2 bytes) 2024-09-17T22:57:42.901 INFO:teuthology.orchestra.run.smithi159.stdout:1543 read 0x43e152 thru 0x44cfb5 (0xee64 bytes) 2024-09-17T22:57:42.903 INFO:teuthology.orchestra.run.smithi159.stdout:1544 write 0x229f7af thru 0x22a7e52 (0x86a4 bytes) 2024-09-17T22:57:42.906 INFO:teuthology.orchestra.run.smithi159.stdout:1545 read 0x183f330 thru 0x1843911 (0x45e2 bytes) 2024-09-17T22:57:42.906 INFO:teuthology.orchestra.run.smithi159.stdout:1546 write 0x7772546 thru 0x777c41d (0x9ed8 bytes) 2024-09-17T22:57:42.910 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:42.907+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:42.927 INFO:teuthology.orchestra.run.smithi159.stdout:1547 writesame 0xe79cb74 thru 0xe7a5855 data_size 0x3(0x8ce2 bytes) 2024-09-17T22:57:42.927 INFO:teuthology.orchestra.run.smithi159.stdout:1549 punch from 0xe117eb1 to 0xe117fdc, (0x12b bytes) 2024-09-17T22:57:42.927 INFO:teuthology.orchestra.run.smithi159.stdout:1550 read 0xd4e8215 thru 0xd4f20b9 (0x9ea5 bytes) 2024-09-17T22:57:42.927 INFO:teuthology.orchestra.run.smithi159.stdout:1554 write 0x1500f6b thru 0x150dc9b (0xcd31 bytes) 2024-09-17T22:57:42.941 INFO:teuthology.orchestra.run.smithi159.stdout:1555 trunc from 0xe7a5856 to 0xc4e6b12 2024-09-17T22:57:42.947 INFO:teuthology.orchestra.run.smithi159.stdout:1557 write 0xad35d68 thru 0xad3a6ce (0x4967 bytes) 2024-09-17T22:57:42.955 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:42.955 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:42.957 INFO:teuthology.orchestra.run.smithi159.stdout:1558 read 0x584ed39 thru 0x5851cfb (0x2fc3 bytes) 2024-09-17T22:57:42.957 INFO:teuthology.orchestra.run.smithi159.stdout:1560 write 0x5204680 thru 0x52062d8 (0x1c59 bytes) 2024-09-17T22:57:42.960 INFO:teuthology.orchestra.run.smithi159.stdout:1561 read 0x20a6d1a thru 0x20b11fa (0xa4e1 bytes) 2024-09-17T22:57:42.960 INFO:teuthology.orchestra.run.smithi159.stdout:1562 read 0x6f4bb2 thru 0x6ffd69 (0xb1b8 bytes) 2024-09-17T22:57:42.961 INFO:teuthology.orchestra.run.smithi159.stdout:1563 punch from 0x744e61b to 0x745ba56, (0xd43b bytes) 2024-09-17T22:57:42.962 INFO:teuthology.orchestra.run.smithi159.stdout:1564 read 0x89be00e thru 0x89c95ec (0xb5df bytes) 2024-09-17T22:57:42.962 INFO:teuthology.orchestra.run.smithi159.stdout:1565 writesame 0x797e647 thru 0x798a8dc data_size 0x1(0xc296 bytes) 2024-09-17T22:57:42.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:42.979 INFO:teuthology.orchestra.run.smithi159.stdout:1566 punch from 0xb2a5468 to 0xb2b32bb, (0xde53 bytes) 2024-09-17T22:57:42.979 INFO:teuthology.orchestra.run.smithi159.stdout:1567 punch from 0x3691364 to 0x369dc04, (0xc8a0 bytes) 2024-09-17T22:57:42.979 INFO:teuthology.orchestra.run.smithi159.stdout:1568 write 0xb915a0 thru 0xba0b84 (0xf5e5 bytes) 2024-09-17T22:57:43.000 INFO:teuthology.orchestra.run.smithi159.stdout:1569 read 0x98cc280 thru 0x98dbbaa (0xf92b bytes) 2024-09-17T22:57:43.001 INFO:teuthology.orchestra.run.smithi159.stdout:1571 trunc from 0xc4e6b12 to 0xc7c82fd 2024-09-17T22:57:43.007 INFO:teuthology.orchestra.run.smithi159.stdout:1572 read 0x2f236da thru 0x2f2656b (0x2e92 bytes) 2024-09-17T22:57:43.011 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:43.008+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:43.015 INFO:teuthology.orchestra.run.smithi159.stdout:1573 write 0xedf935c thru 0xedff4bc (0x6161 bytes) 2024-09-17T22:57:43.020 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:43.020 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:43.020 INFO:teuthology.orchestra.run.smithi159.stdout:1576 read 0xd01376c thru 0xd01cb6c (0x9401 bytes) 2024-09-17T22:57:43.035 INFO:teuthology.orchestra.run.smithi159.stdout:1577 trunc from 0xedff4bd to 0x2478108 2024-09-17T22:57:43.067 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:57:43.095 INFO:teuthology.orchestra.run.smithi159.stdout:1581 writesame 0x411e3d1 thru 0x41226fa data_size 0x1(0x432a bytes) 2024-09-17T22:57:43.099 INFO:teuthology.orchestra.run.smithi159.stdout:1582 trunc from 0x41226fb to 0xd05f88 2024-09-17T22:57:43.101 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:43.101 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:43.113 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:43.110+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:43.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:43.141 INFO:teuthology.orchestra.run.smithi159.stdout:1583 trunc from 0xd05f88 to 0xa243780 2024-09-17T22:57:43.148 INFO:teuthology.orchestra.run.smithi159.stdout:1584 write 0x2dd83fc thru 0x2dd898d (0x592 bytes) 2024-09-17T22:57:43.152 INFO:teuthology.orchestra.run.smithi159.stdout:1586 read 0x26b8714 thru 0x26ba17a (0x1a67 bytes) 2024-09-17T22:57:43.152 INFO:teuthology.orchestra.run.smithi159.stdout:1587 read 0x623f2ec thru 0x624e012 (0xed27 bytes) 2024-09-17T22:57:43.158 INFO:teuthology.orchestra.run.smithi159.stdout:1588 write 0xb41816b thru 0xb41c14e (0x3fe4 bytes) 2024-09-17T22:57:43.165 INFO:teuthology.orchestra.run.smithi159.stdout:1589 writesame 0x8ab81f3 thru 0x8abc3d4 data_size 0x3(0x41e2 bytes) 2024-09-17T22:57:43.171 INFO:teuthology.orchestra.run.smithi159.stdout:1590 punch from 0x6ef0019 to 0x6ef9b8d, (0x9b74 bytes) 2024-09-17T22:57:43.171 INFO:teuthology.orchestra.run.smithi159.stdout:1591 writesame 0x9153d2a thru 0x915cdb4 data_size 0x1(0x908b bytes) 2024-09-17T22:57:43.199 INFO:teuthology.orchestra.run.smithi159.stdout:1592 write 0xde32c26 thru 0xde3af2f (0x830a bytes) 2024-09-17T22:57:43.209 INFO:teuthology.orchestra.run.smithi159.stdout:1593 writesame 0x6128077 thru 0x6134eaf data_size 0x1f(0xce39 bytes) 2024-09-17T22:57:43.213 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:43.211+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:43.219 INFO:teuthology.orchestra.run.smithi159.stdout:1594 punch from 0x1991e38 to 0x19938ef, (0x1ab7 bytes) 2024-09-17T22:57:43.231 INFO:teuthology.orchestra.run.smithi159.stdout:1596 trunc from 0xde3af30 to 0x569e2d8 2024-09-17T22:57:43.255 INFO:teuthology.orchestra.run.smithi159.stdout:1598 write 0x99ca057 thru 0x99cb2db (0x1285 bytes) 2024-09-17T22:57:43.258 INFO:teuthology.orchestra.run.smithi159.stdout:1599 punch from 0x8a8a001 to 0x8a98438, (0xe437 bytes) 2024-09-17T22:57:43.258 INFO:teuthology.orchestra.run.smithi159.stdout:1600 punch from 0x79c3648 to 0x79c9f47, (0x68ff bytes) 2024-09-17T22:57:43.258 INFO:teuthology.orchestra.run.smithi159.stdout:1601 write 0x7cf0df9 thru 0x7cfa8a9 (0x9ab1 bytes) 2024-09-17T22:57:43.266 INFO:teuthology.orchestra.run.smithi159.stdout:1602 clone 15 order 20 su 262144 sc 5 2024-09-17T22:57:43.272 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:43.272 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:43.281 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:43.322 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:43.320+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:43.362 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:43.416 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:43.413+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:43.427 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:43.427 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:43.436 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:43.489 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:43.489 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:43.517 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:44.133 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:44.133 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:43.515+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:44.133 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:44.133 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:44.134 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:44.134 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:44.135 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.135 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:44.135 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:44.135 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:44.136 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:44.136 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:44.136 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:44.136 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:57:44.137 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:57:44.137 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:44.138 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:44.138 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:44.139 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:44.139 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone14 from 0x99cb2dc (overlap 0xd05f88) to 0x535969 2024-09-17T22:57:44.139 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:44.136+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:44.239 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:44.236+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:44.255 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T22:57:44.321 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.321 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:44.321 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:44.321 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:44.321 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:44.331 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:44.340 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:57:44.337+0000 7f2dad65a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33191) UID: 0 2024-09-17T22:57:44.356 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:44.356 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:44.357 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:44.357 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T22:57:44.357 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:57:44.392 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T22:57:44.393 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-17T22:57:44.393 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T22:57:44.393 DEBUG:teuthology.orchestra.run.smithi171:> type ceph-objectstore-tool 2024-09-17T22:57:44.409 INFO:teuthology.orchestra.run.smithi159.stdout:1603 write 0x7a54a2a thru 0x7a59270 (0x4847 bytes) 2024-09-17T22:57:44.420 INFO:teuthology.orchestra.run.smithi159.stdout:1604 punch from 0x562a31c to 0x56364e4, (0xc1c8 bytes) 2024-09-17T22:57:44.420 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:44.420 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:44.421 INFO:teuthology.orchestra.run.smithi159.stdout:1609 read 0x76da311 thru 0x76e75f1 (0xd2e1 bytes) 2024-09-17T22:57:44.421 INFO:teuthology.orchestra.run.smithi159.stdout:1610 read 0x27b7951 thru 0x27b8477 (0xb27 bytes) 2024-09-17T22:57:44.421 INFO:teuthology.orchestra.run.smithi159.stdout:1611 read 0x655c824 thru 0x65669ac (0xa189 bytes) 2024-09-17T22:57:44.421 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:44.421 INFO:teuthology.orchestra.run.smithi159.stdout:1613 write 0x730d9c5 thru 0x731d778 (0xfdb4 bytes) 2024-09-17T22:57:44.435 INFO:teuthology.orchestra.run.smithi159.stdout:1614 write 0xebe790c thru 0xebf64ca (0xebbf bytes) 2024-09-17T22:57:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:1615 writesame 0xe42d056 thru 0xe437427 data_size 0xd(0xa3d2 bytes) 2024-09-17T22:57:44.440 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:44.438+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.442 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16134 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1840203739999993, 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036277120000000003, 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16134 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255129+0000", 2024-09-17T22:57:44.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255131+0000", 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.916e-06 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255137+0000", 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3509999999999998e-06 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255142+0000", 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.882e-06 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291253+0000", 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036111153 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291337+0000", 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3535000000000006e-05 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291405+0000", 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8858999999999999e-05 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16135 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1840004010000005, 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036429299999999998, 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16135 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.448 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255149+0000", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255151+0000", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255157+0000", 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:57:44.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255162+0000", 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291437+0000", 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036274726 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291514+0000", 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7342000000000006e-05 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291578+0000", 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3634999999999996e-05 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16159 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.452 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1839806619999997, 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036685328000000003, 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16159 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255170+0000", 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:44.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255177+0000", 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4989999999999999e-06 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255193+0000", 2024-09-17T22:57:44.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909999999999998e-05 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291665+0000", 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036471816999999997 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291751+0000", 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6509999999999997e-05 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291853+0000", 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102099 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16161 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1839431109999996, 2024-09-17T22:57:44.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036865504, 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16161 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:44.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255208+0000", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3250000000000002e-06 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255218+0000", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4080000000000004e-06 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255223+0000", 2024-09-17T22:57:44.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291884+0000", 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036661457000000001 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291966+0000", 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1920000000000002e-05 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292071+0000", 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000104608 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16163 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:44.463 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1839186609999999, 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037090170999999998, 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16163 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:44.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255233+0000", 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255242+0000", 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0750000000000004e-06 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255247+0000", 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292101+0000", 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036854162000000003 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292174+0000", 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2805000000000005e-05 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292320+0000", 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014646799999999999 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16165 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1728512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1838952230000004, 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037305086000000001, 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.469 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16165 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255254+0000", 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255256+0000", 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.12e-06 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255265+0000", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8959999999999992e-06 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255270+0000", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8760000000000004e-06 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292365+0000", 2024-09-17T22:57:44.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037095377999999998 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292452+0000", 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6947000000000006e-05 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292558+0000", 2024-09-17T22:57:44.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010643 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16167 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1838721870000004, 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037507133999999998, 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.474 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16167 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255277+0000", 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255279+0000", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255288+0000", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8429999999999999e-06 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255293+0000", 2024-09-17T22:57:44.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292603+0000", 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037309869000000002 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292681+0000", 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8484000000000003e-05 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292784+0000", 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010224 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16169 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1838491659999999, 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037775437000000002, 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16169 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255300+0000", 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255302+0000", 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255308+0000", 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255313+0000", 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292863+0000", 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037549224999999999 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292969+0000", 2024-09-17T22:57:44.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010604500000000001 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293075+0000", 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010620699999999999 2024-09-17T22:57:44.485 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16171 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1875968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:44.486 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1838292720000005, 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037958219000000001, 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.487 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16171 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:44.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255320+0000", 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2899999999999999e-07 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255322+0000", 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.906e-06 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255337+0000", 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5452999999999999e-05 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255343+0000", 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:57:44.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293106+0000", 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037763707000000001 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293186+0000", 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0153000000000003e-05 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293278+0000", 2024-09-17T22:57:44.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1114999999999998e-05 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16172 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1925120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1837989889999996, 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038145196999999999, 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16172 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8400000000000004e-07 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255359+0000", 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:44.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255367+0000", 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0649999999999999e-06 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255372+0000", 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1390000000000003e-06 2024-09-17T22:57:44.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293318+0000", 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037945566 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293399+0000", 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1168999999999995e-05 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293495+0000", 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6162000000000005e-05 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.502 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16173 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1837677939999995, 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038380816999999998, 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.503 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16173 2024-09-17T22:57:44.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255388+0000", 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255402+0000", 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4532000000000001e-05 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255408+0000", 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7740000000000001e-06 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293577+0000", 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038169526000000002 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293664+0000", 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6312999999999998e-05 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293762+0000", 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7947000000000003e-05 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.508 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16174 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1837330379999997, 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038531986999999997, 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16174 2024-09-17T22:57:44.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255418+0000", 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255426+0000", 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1620000000000002e-06 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255436+0000", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0851e-05 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293804+0000", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038367215000000003 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293886+0000", 2024-09-17T22:57:44.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2189999999999997e-05 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293948+0000", 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1709000000000002e-05 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16176 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1837050849999997, 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038738529000000001, 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16176 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255446+0000", 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255453+0000", 2024-09-17T22:57:44.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8170000000000001e-06 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255458+0000", 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293991+0000", 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038532376 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294084+0000", 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3614000000000002e-05 2024-09-17T22:57:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294182+0000", 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7949999999999996e-05 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16239 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1803326260000002, 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036338338999999997, 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16239 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258819+0000", 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3e-06 2024-09-17T22:57:44.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258825+0000", 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258830+0000", 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294968+0000", 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036137423000000002 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295061+0000", 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3177999999999995e-05 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295154+0000", 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3645000000000002e-05 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16241 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1835008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1803080599999998, 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036566098999999998, 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16241 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4770000000000002e-06 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258850+0000", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5490000000000003e-06 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258855+0000", 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.904e-06 2024-09-17T22:57:44.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295247+0000", 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036392378000000003 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295334+0000", 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7058999999999994e-05 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295407+0000", 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2287999999999996e-05 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16243 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1802861950000008, 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036705072999999998, 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.532 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16243 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:44.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258863+0000", 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258866+0000", 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.678e-06 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258872+0000", 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258877+0000", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295445+0000", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036567582000000001 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295506+0000", 2024-09-17T22:57:44.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1895000000000003e-05 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295568+0000", 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1085000000000002e-05 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.537 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16245 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1933312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1799596090000009, 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036671423000000002, 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16245 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259190+0000", 2024-09-17T22:57:44.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259194+0000", 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259205+0000", 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0468e-05 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259212+0000", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1380000000000001e-06 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295666+0000", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036453700999999998 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295753+0000", 2024-09-17T22:57:44.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7321e-05 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295860+0000", 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010761599999999999 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16246 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:44.542 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1799250519999998, 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036882379, 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16246 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:44.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6599999999999996e-07 2024-09-17T22:57:44.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259227+0000", 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259234+0000", 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8539999999999999e-06 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259239+0000", 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295903+0000", 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036664018999999999 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295993+0000", 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9944000000000005e-05 2024-09-17T22:57:44.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296106+0000", 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113158 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16248 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:44.547 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1799028820000004, 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037070651000000003, 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16248 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:44.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259248+0000", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259254+0000", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259260+0000", 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4489999999999998e-06 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296156+0000", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036896089999999999 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296238+0000", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2616999999999999e-05 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296316+0000", 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7964000000000001e-05 2024-09-17T22:57:44.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16249 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.1797319940000008, 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037100148999999999, 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:44.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16249 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8700000000000005e-07 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259421+0000", 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0470000000000004e-06 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259431+0000", 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4190000000000008e-06 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259437+0000", 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4520000000000002e-06 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296352+0000", 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036914354000000003 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296427+0000", 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5603000000000001e-05 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296517+0000", 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9586999999999995e-05 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.557 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:44.559 INFO:teuthology.orchestra.run.smithi159.stdout:1617 read 0x2a49af8 thru 0x2a57902 (0xde0b bytes) 2024-09-17T22:57:44.559 INFO:teuthology.orchestra.run.smithi159.stdout:1618 write 0xb49e1a0 thru 0xb49f7ba (0x161b bytes) 2024-09-17T22:57:44.559 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:57:44.560 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-17T22:57:44.560 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:44.556+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:44.561 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:44.660 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:44.657+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:44.705 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.705 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:44.705 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:44.705 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:44.714 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:44.761 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:44.758+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:44.818 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:44.818 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:44.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:44.868 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:44.865+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:44.962 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:44.960+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:44.971 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.077191081999999, 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.975 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1044 3.9 e28/22 3:954c0790:::rbd_data.118840f89003.0000000000000090:head v 28'4, mlcod=25'2)", 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.939389378000001, 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032794199999999999, 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.09e-07 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028800+0000", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7649999999999999e-05 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028812+0000", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1865000000000001e-05 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028819+0000", 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7750000000000004e-06 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028870+0000", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0949000000000003e-05 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028889+0000", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9199999999999999e-05 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032019+0000", 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031302230000000001 2024-09-17T22:57:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032061+0000", 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1949000000000001e-05 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 41 entries 28'5 (0'0) error 3:9656af7d:::rbd_data.118840f89003.0000000000000092:head by client.4488.0:1048 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.939316230999999, 2024-09-17T22:57:44.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034247520000000001, 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:44.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028863+0000", 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7249999999999997e-06 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028879+0000", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5862000000000002e-05 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028892+0000", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3076e-05 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.029115+0000", 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000223427 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032279+0000", 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.003164189 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.4 epoch 28/22 rep_tid 35 entries 28'5 (0'0) error 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head by client.4488.0:1103 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'3)", 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.929607399999998, 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00299241, 2024-09-17T22:57:44.984 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6199999999999997e-07 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038581+0000", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7116e-05 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038603+0000", 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.177e-05 2024-09-17T22:57:44.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038623+0000", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9772e-05 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038712+0000", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8910000000000001e-05 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.041556+0000", 2024-09-17T22:57:44.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028440800000000001 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 50 entries 28'7 (0'0) error 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head by client.4488.0:1110 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.928714105000001, 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0077516210000000002, 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:44.988 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039458+0000", 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.99e-07 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039466+0000", 2024-09-17T22:57:44.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4419999999999998e-06 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039486+0000", 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0523999999999999e-05 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039502+0000", 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5930000000000002e-05 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039590+0000", 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8012999999999999e-05 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047208+0000", 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0076180129999999999 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1121 3.4 e28/22 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head v 28'6, mlcod=27'4)", 2024-09-17T22:57:44.993 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.925611400000001, 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004866296, 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042566+0000", 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042575+0000", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3349999999999996e-06 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042580+0000", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042623+0000", 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3396000000000003e-05 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042644+0000", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1155999999999999e-05 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047379+0000", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047352699999999998 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047426+0000", 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6424999999999997e-05 2024-09-17T22:57:44.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:5169 3.9 e32/22 3:92748aee:::rbd_data.118846a2cbc0.0000000000000049:head v 32'12, mlcod=28'10)", 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.395551016000001, 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029730630000000002, 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:44.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572620+0000", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572621+0000", 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.561e-06 2024-09-17T22:57:44.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572630+0000", 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4020000000000002e-06 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572649+0000", 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9216999999999999e-05 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572658+0000", 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5879999999999998e-06 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572723+0000", 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5339999999999994e-05 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.572746+0000", 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3003e-05 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575556+0000", 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028103970000000001 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.575593+0000", 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6554999999999998e-05 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6635 3.4 e32/22 3:264a9651:::rbd_data.118846a2cbc0.0000000000000014:head v 32'17, mlcod=32'15)", 2024-09-17T22:57:45.002 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.035003536, 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031534419999999998, 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933168+0000", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933175+0000", 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8179999999999998e-06 2024-09-17T22:57:45.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933185+0000", 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0155e-05 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933191+0000", 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0499999999999997e-06 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933235+0000", 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3951e-05 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933250+0000", 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4605e-05 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936290+0000", 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030402879999999999 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936321+0000", 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0905999999999999e-05 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6774 3.0 e33/22 3:05c44946:::rbd_data.118846a2cbc0.0000000000000045:head v 33'36, mlcod=32'33)", 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.892726353, 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053547780000000001, 2024-09-17T22:57:45.007 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0299999999999998e-07 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075451+0000", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075468+0000", 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7212999999999999e-05 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075474+0000", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075522+0000", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8114e-05 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075543+0000", 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0389000000000001e-05 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080753+0000", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052103440000000004 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080799+0000", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6335999999999998e-05 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.11 epoch 33/28 rep_tid 246 entries 33'38 (0'0) error 3:89c4a1e2:::rbd_header.118846a2cbc0:head by client.4488.0:7335 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'36)", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:45.011 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.773248977, 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023844890000000001, 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194922+0000", 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.06e-07 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194929+0000", 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1770000000000003e-06 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194948+0000", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9021999999999998e-05 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.194960+0000", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2248e-05 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195007+0000", 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7064000000000002e-05 2024-09-17T22:57:45.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.197306+0000", 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022994719999999999 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.015 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7337 3.11 e33/28 3:89c4a1e2:::rbd_header.118846a2cbc0:head v 33'39, mlcod=33'36)", 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.77288274, 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046186769999999999, 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195289+0000", 2024-09-17T22:57:45.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195293+0000", 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.155e-06 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195301+0000", 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8520000000000004e-06 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195307+0000", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195340+0000", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3090999999999999e-05 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195353+0000", 2024-09-17T22:57:45.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2757e-05 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199876+0000", 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045229500000000004 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199907+0000", 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1340000000000001e-05 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14579 3.11 e37/28 3:8ec16ab6:::rbd_data.11889f90e108.0000000000000148:head v 37'73, mlcod=36'71)", 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.8891379560000008, 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002650075, 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079033+0000", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079035+0000", 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7239999999999999e-06 2024-09-17T22:57:45.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079044+0000", 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3780000000000007e-06 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079063+0000", 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9412999999999999e-05 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079074+0000", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0366999999999999e-05 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079135+0000", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1362e-05 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.079158+0000", 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2388e-05 2024-09-17T22:57:45.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081639+0000", 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002481797 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081683+0000", 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3646000000000002e-05 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14587 3.4 e37/22 3:23430168:::rbd_data.11889f90e108.000000000000021d:head v 37'90, mlcod=37'88)", 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.8865119359999998, 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002639691, 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:45.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081659+0000", 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081660+0000", 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6599999999999996e-07 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081666+0000", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081675+0000", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5279999999999992e-06 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081682+0000", 2024-09-17T22:57:45.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7129999999999999e-06 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081730+0000", 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8174000000000003e-05 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.081749+0000", 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9279999999999998e-05 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084252+0000", 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025027280000000001 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084299+0000", 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6601000000000003e-05 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.030 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:14599 3.0 e37/22 3:03d9ef58:::rbd_data.11889f90e108.000000000000035d:head v 37'60, mlcod=37'58)", 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.8835317430000007, 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023997279999999998, 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:45.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084639+0000", 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084640+0000", 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5799999999999998e-07 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084646+0000", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2299999999999996e-06 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084657+0000", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1137e-05 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084664+0000", 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:57:45.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084719+0000", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5336e-05 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.084738+0000", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9122000000000001e-05 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.086996+0000", 2024-09-17T22:57:45.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022571259999999999 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:35.087039+0000", 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3290000000000001e-05 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22674 3.0 e45/22 3:000cde75:::rbd_data.1188903913c1.000000000000001f:head v 45'77, mlcod=44'74)", 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.82822732300000002, 2024-09-17T22:57:45.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0062289370000000004, 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:45.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139953+0000", 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0229999999999998e-06 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139962+0000", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-06 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139968+0000", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140030+0000", 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1803000000000004e-05 2024-09-17T22:57:45.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140046+0000", 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6093999999999999e-05 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146129+0000", 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0060834239999999996 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146173+0000", 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3238999999999997e-05 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.0 epoch 45/22 rep_tid 301 entries 45'78 (0'0) error 3:047d1ef5:::rbd_data.1188903913c1.0000000000000029:head by client.4488.0:22684 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 44'74)", 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.826529085, 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.007471218, 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.44e-07 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141651+0000", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4449999999999996e-06 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141671+0000", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0373e-05 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141695+0000", 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4224e-05 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141782+0000", 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6514999999999994e-05 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149113+0000", 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0073311169999999998 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.043 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22687 3.4 e45/22 3:25296614:::rbd_data.1188903913c1.000000000000002b:head v 45'156, mlcod=44'152)", 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.82565547100000003, 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0066082959999999996, 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142516+0000", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3900000000000004e-07 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142523+0000", 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7429999999999996e-06 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142536+0000", 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3151e-05 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142543+0000", 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7100000000000001e-06 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142607+0000", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3765999999999999e-05 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142627+0000", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.035e-05 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149088+0000", 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0064607140000000002 2024-09-17T22:57:45.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149124+0000", 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6123e-05 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22853 3.11 e45/28 3:8e31397f:::rbd_data.1188903913c1.000000000000002f:head v 45'102, mlcod=44'99)", 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.147037+0000", 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.82113383799999995, 2024-09-17T22:57:45.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002485335, 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147037+0000", 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147037+0000", 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147037+0000", 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.03e-07 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147052+0000", 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4457999999999999e-05 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147062+0000", 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0030999999999999e-05 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147069+0000", 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5959999999999998e-06 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147134+0000", 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4591000000000006e-05 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.147156+0000", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2388999999999999e-05 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149480+0000", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002323121 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149522+0000", 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2846000000000003e-05 2024-09-17T22:57:45.052 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22900 3.0 e45/22 3:047d1ef5:::rbd_data.1188903913c1.0000000000000029:head v 45'81, mlcod=45'78)", 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.152692+0000", 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.815478916, 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0027087040000000001, 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152692+0000", 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152692+0000", 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152693+0000", 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3099999999999996e-07 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152699+0000", 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2260000000000002e-06 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152709+0000", 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0154e-05 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152715+0000", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8629999999999996e-06 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152760+0000", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.481e-05 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.152775+0000", 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5412000000000001e-05 2024-09-17T22:57:45.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.155361+0000", 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002585295 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.155401+0000", 2024-09-17T22:57:45.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0113e-05 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22907 3.4 e45/22 3:244fb18a:::rbd_data.1188903913c1.00000000000001f6:head v 45'158, mlcod=45'154)", 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.80908255799999995, 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031838320000000002, 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159089+0000", 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5499999999999998e-07 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159095+0000", 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4409999999999998e-06 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159108+0000", 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.258e-05 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159115+0000", 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9419999999999998e-06 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159210+0000", 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4853999999999997e-05 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159231+0000", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1583e-05 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162234+0000", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030021539999999999 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162272+0000", 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8622999999999999e-05 2024-09-17T22:57:45.062 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.063 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:45.064 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:45.065 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:45.062+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:45.087 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:57:45.165 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:45.162+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:45.212 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:45.212 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:45.212 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:45.212 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:45.221 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:45.266 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:45.263+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:45.326 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:45.326 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:45.334 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:45.378 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:45.376+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.439 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5757 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849364616999999, 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038199444999999999, 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5757 2024-09-17T22:57:45.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:45.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5600000000000002e-07 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586464+0000", 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586468+0000", 2024-09-17T22:57:45.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8630000000000001e-06 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586471+0000", 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7549999999999999e-06 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624560+0000", 2024-09-17T22:57:45.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038088960999999998 2024-09-17T22:57:45.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624607+0000", 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6740000000000003e-05 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624661+0000", 2024-09-17T22:57:45.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4732999999999999e-05 2024-09-17T22:57:45.449 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.449 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5758 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:45.450 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849351728, 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038315071999999999, 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.451 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5758 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:45.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:45.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5100000000000001e-07 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586476+0000", 2024-09-17T22:57:45.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.113e-06 2024-09-17T22:57:45.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586483+0000", 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5200000000000003e-06 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586487+0000", 2024-09-17T22:57:45.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3479999999999997e-06 2024-09-17T22:57:45.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624681+0000", 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038194371999999997 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624722+0000", 2024-09-17T22:57:45.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0367999999999997e-05 2024-09-17T22:57:45.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624790+0000", 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8100000000000002e-05 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.462 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5759 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3948544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.462 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:45.462 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849336002999999, 2024-09-17T22:57:45.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038450864000000001, 2024-09-17T22:57:45.462 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.463 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5759 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:45.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:45.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586491+0000", 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:57:45.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586492+0000", 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5349999999999999e-06 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586495+0000", 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586498+0000", 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6170000000000001e-06 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624828+0000", 2024-09-17T22:57:45.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038329514000000002 2024-09-17T22:57:45.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624875+0000", 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7107999999999997e-05 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624941+0000", 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6647000000000001e-05 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5760 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849325296, 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038669935000000003, 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.472 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5760 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586503+0000", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0899999999999999e-06 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586506+0000", 2024-09-17T22:57:45.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586508+0000", 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624980+0000", 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038472343999999999 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625076+0000", 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5214e-05 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625171+0000", 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5421999999999995e-05 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5761 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4276224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:45.476 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849315316, 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038805352000000001, 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5761 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.22e-07 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586512+0000", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0210000000000001e-06 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586515+0000", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586518+0000", 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:57:45.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625211+0000", 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038693268000000003 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625261+0000", 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9345000000000003e-05 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625317+0000", 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5850999999999998e-05 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5762 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4472832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849305425000001, 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038916981000000003, 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.481 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5762 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:45.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586525+0000", 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.224e-06 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586532+0000", 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3950000000000003e-06 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586535+0000", 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625339+0000", 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038804584000000003 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625385+0000", 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5099000000000003e-05 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625438+0000", 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3511999999999999e-05 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.485 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5763 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4669440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849287495, 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039029253999999999, 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5763 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586541+0000", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.268e-06 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586547+0000", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586554+0000", 2024-09-17T22:57:45.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625465+0000", 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038910924 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625511+0000", 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5179999999999998e-05 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625568+0000", 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7738999999999997e-05 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5764 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4866048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849266644, 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039122912000000003, 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.490 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5764 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:45.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586561+0000", 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.266e-06 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586566+0000", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586568+0000", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625592+0000", 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039024174000000002 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625634+0000", 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1464000000000001e-05 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625683+0000", 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9002000000000003e-05 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.494 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5765 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5062656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.849254797, 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039240865999999999, 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5765 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586895+0000", 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000323658 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586899+0000", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6260000000000001e-06 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586910+0000", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1239e-05 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586917+0000", 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:57:45.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625710+0000", 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038792608999999999 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625757+0000", 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7416000000000001e-05 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625813+0000", 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5395000000000001e-05 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5766 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5259264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.848898665, 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038992026999999999, 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.499 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5766 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586931+0000", 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586939+0000", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2549999999999997e-06 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586945+0000", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625828+0000", 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038882947000000001 2024-09-17T22:57:45.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625871+0000", 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3204000000000002e-05 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625920+0000", 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8539999999999999e-05 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.503 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5767 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5455872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.848873575000001, 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039097260000000002, 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5767 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588020+0000", 2024-09-17T22:57:45.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00106675 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588024+0000", 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588033+0000", 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588044+0000", 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1800000000000001e-05 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625943+0000", 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037898695000000003 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625988+0000", 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4577999999999999e-05 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626050+0000", 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2605000000000001e-05 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5769 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5849088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.847742943, 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038207729000000003, 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.508 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5769 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588086+0000", 2024-09-17T22:57:45.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.103e-06 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588094+0000", 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6459999999999993e-06 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588099+0000", 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1259999999999997e-06 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626190+0000", 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.03809067 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626234+0000", 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4051999999999998e-05 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626291+0000", 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7694e-05 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.512 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5770 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.847719971, 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038303865999999999, 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.513 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5770 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588109+0000", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0899999999999999e-06 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588116+0000", 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2439999999999996e-06 2024-09-17T22:57:45.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588122+0000", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8640000000000001e-06 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626315+0000", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038193105999999998 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626359+0000", 2024-09-17T22:57:45.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3828000000000001e-05 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626410+0000", 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1328000000000001e-05 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5771 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.847697691, 2024-09-17T22:57:45.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038402700999999997, 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5771 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588131+0000", 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588138+0000", 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3049999999999998e-06 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588143+0000", 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4650000000000002e-06 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626433+0000", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038289895999999997 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626476+0000", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2913000000000001e-05 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626532+0000", 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5692000000000001e-05 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.521 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5772 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6438912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.847676033000001, 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038494234000000002, 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5772 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590017+0000", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018664980000000001 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590020+0000", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4769999999999999e-06 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590027+0000", 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5209999999999999e-06 2024-09-17T22:57:45.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590032+0000", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626554+0000", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036521551999999999 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626595+0000", 2024-09-17T22:57:45.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1900000000000002e-05 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626645+0000", 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9205000000000001e-05 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5828 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.845482626000001, 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038160161999999997, 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5828 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.527 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:45.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4600000000000001e-07 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590345+0000", 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.376e-06 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590349+0000", 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590352+0000", 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7410000000000001e-06 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628400+0000", 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038048347000000003 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628448+0000", 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7961999999999999e-05 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628504+0000", 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6137000000000001e-05 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5829 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.845471571000001, 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038295726000000002, 2024-09-17T22:57:45.531 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5829 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592028+0000", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001673379 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592032+0000", 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5109999999999999e-06 2024-09-17T22:57:45.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592038+0000", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7379999999999999e-06 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592042+0000", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628543+0000", 2024-09-17T22:57:45.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036500828999999999 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628594+0000", 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1737999999999998e-05 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628651+0000", 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6320999999999998e-05 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5840 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.843620936000001, 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038232347, 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5840 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.536 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:45.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592208+0000", 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-06 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592214+0000", 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7479999999999999e-06 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592220+0000", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.037e-06 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630310+0000", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038090474999999999 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630373+0000", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2467999999999999e-05 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630438+0000", 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5235999999999997e-05 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5841 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3375104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.843600697999999, 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038361579999999999, 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.540 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5841 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592228+0000", 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592235+0000", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8040000000000004e-06 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592242+0000", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2230000000000001e-06 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630468+0000", 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038225982999999998 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630517+0000", 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8656000000000003e-05 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630587+0000", 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0698000000000006e-05 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5842 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.843577968, 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038469774999999998, 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5842 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:45.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594021+0000", 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001772849 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594026+0000", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4399999999999998e-06 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594037+0000", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1071e-05 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594044+0000", 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9129999999999997e-06 2024-09-17T22:57:45.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630613+0000", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036569602999999999 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630659+0000", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5246e-05 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630718+0000", 2024-09-17T22:57:45.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9653e-05 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:45.549 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:45.550 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:45.551 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:45.548+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:45.651 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:45.648+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:45.664 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:45.664 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:45.664 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:45.672 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:45.752 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:45.750+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:45.817 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:45.818 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:45.818 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:45.827 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:45.853 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:45.851+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.932 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7313 3.1a 3:5926c794:::rbd_data.118846a2cbc0.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.132345+0000", 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.796728582, 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.011501466, 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7313 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.933 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.132345+0000", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.132345+0000", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.132351+0000", 2024-09-17T22:57:45.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134743+0000", 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023927079999999999 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134756+0000", 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2174000000000001e-05 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134763+0000", 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6809999999999998e-06 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134822+0000", 2024-09-17T22:57:45.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8686e-05 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134961+0000", 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000138825 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135269+0000", 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030812599999999998 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.139632+0000", 2024-09-17T22:57:45.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00436353 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.143749+0000", 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041169170000000003 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.143793+0000", 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3559000000000001e-05 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.143846+0000", 2024-09-17T22:57:45.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3094999999999999e-05 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7314 3.1a 3:5926c794:::rbd_data.118846a2cbc0.0000000000000007:head [truncate 278528] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.134785+0000", 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.794288480000001, 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094515490000000001, 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.941 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7314 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134785+0000", 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134785+0000", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134785+0000", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7199999999999997e-07 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134789+0000", 2024-09-17T22:57:45.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0979999999999998e-06 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134796+0000", 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6009999999999997e-06 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134802+0000", 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4929999999999998e-06 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137516+0000", 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027147780000000002 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137580+0000", 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3268999999999993e-05 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137693+0000", 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113662 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:45.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140010+0000", 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002316851 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.144134+0000", 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0041233859999999997 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.144190+0000", 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6329000000000002e-05 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.144236+0000", 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.621e-05 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7317 3.14 3:2e13337d:::rbd_data.118846a2cbc0.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.151654+0000", 2024-09-17T22:57:45.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.777419543000001, 2024-09-17T22:57:45.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010184245999999999, 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7317 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.151654+0000", 2024-09-17T22:57:45.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.151654+0000", 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.151655+0000", 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.11e-06 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.153562+0000", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019075769999999999 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.153573+0000", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1255e-05 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.153579+0000", 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.153626+0000", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7379000000000002e-05 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.153765+0000", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000138379 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.154055+0000", 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029066699999999999 2024-09-17T22:57:45.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.160313+0000", 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062572460000000002 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.161759+0000", 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014462019999999999 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.161802+0000", 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3486999999999999e-05 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.161838+0000", 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5444000000000001e-05 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7321 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.173810+0000", 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.755262627, 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098483790000000009, 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7321 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.173810+0000", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.173810+0000", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.173812+0000", 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.291e-06 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.175678+0000", 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018658539999999999 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.175689+0000", 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1617000000000001e-05 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.175697+0000", 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.559e-06 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.175749+0000", 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2675e-05 2024-09-17T22:57:45.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.175841+0000", 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1416999999999996e-05 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 1", 2024-09-17T22:57:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.176055+0000", 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021460699999999999 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.179869+0000", 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0038139459999999999 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:45.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.183543+0000", 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036733540000000002 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.183607+0000", 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4318000000000003e-05 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.183659+0000", 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1740999999999998e-05 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8126 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 344064~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297370+0000", 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631702676, 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0097600599999999992, 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8126 2024-09-17T22:57:45.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297370+0000", 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297370+0000", 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297371+0000", 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2e-07 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297375+0000", 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-06 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297383+0000", 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9200000000000004e-06 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297387+0000", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-06 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.304601+0000", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0072131269999999997 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.306365+0000", 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001764463 2024-09-17T22:57:45.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307130+0000", 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00076548000000000002 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8127 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 950272~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297395+0000", 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631678287, 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099849789999999997, 2024-09-17T22:57:45.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8127 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297395+0000", 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297395+0000", 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297395+0000", 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297402+0000", 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.951e-06 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297417+0000", 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4782e-05 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297425+0000", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.456e-06 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307161+0000", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0097352489999999996 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307241+0000", 2024-09-17T22:57:45.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9903999999999999e-05 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307380+0000", 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000139238 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8128 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 1572864~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297437+0000", 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631636536, 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010179601, 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8128 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297437+0000", 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297437+0000", 2024-09-17T22:57:45.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297437+0000", 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297440+0000", 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1190000000000001e-06 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297458+0000", 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7873e-05 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297465+0000", 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0770000000000002e-06 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307415+0000", 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.009949734 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307496+0000", 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0661999999999994e-05 2024-09-17T22:57:45.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307616+0000", 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000120628 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8171 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297645+0000", 2024-09-17T22:57:45.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631428519, 2024-09-17T22:57:45.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010182389999999999, 2024-09-17T22:57:45.981 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.981 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.981 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:45.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8171 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297645+0000", 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297645+0000", 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297645+0000", 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297648+0000", 2024-09-17T22:57:45.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7149999999999998e-06 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297654+0000", 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:57:45.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297659+0000", 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8080000000000004e-06 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307645+0000", 2024-09-17T22:57:45.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099862219999999995 2024-09-17T22:57:45.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307740+0000", 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4394999999999993e-05 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:45.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307827+0000", 2024-09-17T22:57:45.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7287999999999995e-05 2024-09-17T22:57:45.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.990 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:45.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:45.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8172 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:45.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297666+0000", 2024-09-17T22:57:45.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631407042999999, 2024-09-17T22:57:45.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010331081000000001, 2024-09-17T22:57:45.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:45.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:45.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:45.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:45.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:45.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:45.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8172 2024-09-17T22:57:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297666+0000", 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297666+0000", 2024-09-17T22:57:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297666+0000", 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.32e-07 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297668+0000", 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0169999999999999e-06 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297675+0000", 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2330000000000003e-06 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297680+0000", 2024-09-17T22:57:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1120000000000004e-06 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307860+0000", 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010179881999999999 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307932+0000", 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2273000000000004e-05 2024-09-17T22:57:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.307997+0000", 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5131999999999999e-05 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8173 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.000 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297687+0000", 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.631386371, 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010436482, 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8173 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297687+0000", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297687+0000", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297687+0000", 2024-09-17T22:57:46.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5100000000000001e-07 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297694+0000", 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3119999999999996e-06 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297701+0000", 2024-09-17T22:57:46.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9700000000000002e-06 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297714+0000", 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2373e-05 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308025+0000", 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010310896 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308072+0000", 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7786e-05 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308123+0000", 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0794000000000002e-05 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8174 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.297722+0000", 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.63135158, 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010629816, 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8174 2024-09-17T22:57:46.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297722+0000", 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297722+0000", 2024-09-17T22:57:46.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297722+0000", 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297724+0000", 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3130000000000001e-06 2024-09-17T22:57:46.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297731+0000", 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8759999999999999e-06 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.297738+0000", 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308169+0000", 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010431930000000001 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308255+0000", 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5217999999999996e-05 2024-09-17T22:57:46.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308351+0000", 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6631000000000004e-05 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8284 3.1a 3:5926c794:::rbd_data.118846a2cbc0.0000000000000007:9 [sparse-read 1048576~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.299463+0000", 2024-09-17T22:57:46.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.629609993999999, 2024-09-17T22:57:46.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0095724590000000002, 2024-09-17T22:57:46.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8284 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299463+0000", 2024-09-17T22:57:46.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299463+0000", 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299463+0000", 2024-09-17T22:57:46.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:57:46.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299465+0000", 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8500000000000001e-06 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299471+0000", 2024-09-17T22:57:46.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.648e-06 2024-09-17T22:57:46.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299476+0000", 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308857+0000", 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093808629999999997 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.308934+0000", 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999994e-05 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309036+0000", 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101298 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.024 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8285 3.1a 3:5926c794:::rbd_data.118846a2cbc0.0000000000000007:9 [sparse-read 1654784~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.024 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.299483+0000", 2024-09-17T22:57:46.024 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.629590460999999, 2024-09-17T22:57:46.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.009789598, 2024-09-17T22:57:46.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.025 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8285 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299483+0000", 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299483+0000", 2024-09-17T22:57:46.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299483+0000", 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299485+0000", 2024-09-17T22:57:46.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-06 2024-09-17T22:57:46.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299491+0000", 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9689999999999999e-06 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299498+0000", 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3280000000000002e-06 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309068+0000", 2024-09-17T22:57:46.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0095698869999999991 2024-09-17T22:57:46.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309144+0000", 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6545999999999995e-05 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309272+0000", 2024-09-17T22:57:46.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012817800000000001 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.034 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8289 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 425984~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.299566+0000", 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.629507534, 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099667509999999994, 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.035 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8289 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.036 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299566+0000", 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299566+0000", 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299566+0000", 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299568+0000", 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299574+0000", 2024-09-17T22:57:46.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9070000000000004e-06 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299579+0000", 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4360000000000004e-06 2024-09-17T22:57:46.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309317+0000", 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0097386480000000008 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309402+0000", 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4338000000000005e-05 2024-09-17T22:57:46.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309532+0000", 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013070399999999999 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8290 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 1048576~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.043 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.299586+0000", 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.629487504, 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010172014, 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8290 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299586+0000", 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299586+0000", 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299586+0000", 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299588+0000", 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1799999999999999e-06 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299594+0000", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5199999999999997e-06 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299598+0000", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309563+0000", 2024-09-17T22:57:46.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099646270000000002 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309640+0000", 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7650000000000004e-05 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309758+0000", 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000117309 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.048 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8291 3.1a 3:5aa7c0c1:::rbd_data.118846a2cbc0.0000000000000009:9 [sparse-read 1654784~81920] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.299604+0000", 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.629468648, 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010378411000000001, 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8291 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299604+0000", 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299604+0000", 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299605+0000", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299607+0000", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.821e-06 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299612+0000", 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:57:46.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.299617+0000", 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7319999999999998e-06 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309785+0000", 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010167819 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309864+0000", 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9555999999999995e-05 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.309983+0000", 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000118422 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8334 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.300303+0000", 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.628769997999999, 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098765520000000002, 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8334 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300303+0000", 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300303+0000", 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300303+0000", 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300312+0000", 2024-09-17T22:57:46.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8009999999999993e-06 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300318+0000", 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300323+0000", 2024-09-17T22:57:46.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9200000000000003e-06 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310012+0000", 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096892220000000008 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310088+0000", 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6119999999999996e-05 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310180+0000", 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1182999999999998e-05 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8335 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.300330+0000", 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.628743101, 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010027499, 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8335 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300330+0000", 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300330+0000", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300330+0000", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.72e-07 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300332+0000", 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9410000000000001e-06 2024-09-17T22:57:46.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300338+0000", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300343+0000", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7899999999999999e-06 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310204+0000", 2024-09-17T22:57:46.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098613390000000002 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310272+0000", 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7533000000000005e-05 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310357+0000", 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5505000000000002e-05 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8336 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.300349+0000", 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.62872437, 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010236604, 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.064 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8336 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300349+0000", 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300349+0000", 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300349+0000", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300351+0000", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.821e-06 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300357+0000", 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7540000000000003e-06 2024-09-17T22:57:46.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300363+0000", 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310404+0000", 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010041374 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310485+0000", 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1212999999999996e-05 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310585+0000", 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000100066 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:8337 3.1a 3:5aa7fe0a:::rbd_data.118846a2cbc0.0000000000000015:9 [sparse-read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.300369+0000", 2024-09-17T22:57:46.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.628703838, 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010435331000000001, 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8337 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:46.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300369+0000", 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300369+0000", 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300370+0000", 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300372+0000", 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9230000000000001e-06 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300378+0000", 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.300382+0000", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6290000000000001e-06 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310618+0000", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010236182999999999 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310703+0000", 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4141000000000006e-05 2024-09-17T22:57:46.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.310805+0000", 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010207799999999999 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:46.073 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:46.075 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:46.075 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:46.072+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:46.076 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:57:46.086 INFO:teuthology.orchestra.run.smithi159.stdout:1620 write 0x72eea35 thru 0x72fb37b (0xc947 bytes) 2024-09-17T22:57:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:1621 read 0xbdc0edd thru 0xbdc8e2c (0x7f50 bytes) 2024-09-17T22:57:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:1624 read 0xde7d679 thru 0xde82806 (0x518e bytes) 2024-09-17T22:57:46.090 INFO:teuthology.orchestra.run.smithi159.stdout:1625 read 0xae8b33e thru 0xae8bc0f (0x8d2 bytes) 2024-09-17T22:57:46.091 INFO:teuthology.orchestra.run.smithi159.stdout:1626 punch from 0x186cba to 0x18dfdd, (0x7323 bytes) 2024-09-17T22:57:46.091 INFO:teuthology.orchestra.run.smithi159.stdout:1629 write 0xfd8ae3 thru 0xfdf33e (0x685c bytes) 2024-09-17T22:57:46.093 INFO:teuthology.orchestra.run.smithi159.stdout:1630 read 0xb3629c thru 0xb3e722 (0x8487 bytes) 2024-09-17T22:57:46.101 INFO:teuthology.orchestra.run.smithi159.stdout:1631 trunc from 0xebf64cb to 0xaed4acc 2024-09-17T22:57:46.110 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.110 INFO:teuthology.orchestra.run.smithi159.stdout:1633 writesame 0x17bae87 thru 0x17c59ba data_size 0x2(0xab34 bytes) 2024-09-17T22:57:46.123 INFO:teuthology.orchestra.run.smithi159.stdout:1635 read 0x58bc168 thru 0x58cc157 (0xfff0 bytes) 2024-09-17T22:57:46.124 INFO:teuthology.orchestra.run.smithi159.stdout:1636 writesame 0x6c0bc44 thru 0x6c179e4 data_size 0x23(0xbda1 bytes) 2024-09-17T22:57:46.128 INFO:teuthology.orchestra.run.smithi159.stdout:1638 write 0x6efe752 thru 0x6f05f5f (0x780e bytes) 2024-09-17T22:57:46.135 INFO:teuthology.orchestra.run.smithi159.stdout:1640 trunc from 0xaed4acc to 0xc8ae58e 2024-09-17T22:57:46.140 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:46.141 INFO:teuthology.orchestra.run.smithi159.stdout:1641 write 0x8982a81 thru 0x89911a0 (0xe720 bytes) 2024-09-17T22:57:46.145 INFO:teuthology.orchestra.run.smithi159.stdout:1643 writesame 0x76f587d thru 0x770258b data_size 0x1(0xcd0f bytes) 2024-09-17T22:57:46.175 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:46.173+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:46.178 INFO:teuthology.orchestra.run.smithi159.stdout:1646 write 0xe055997 thru 0xe062044 (0xc6ae bytes) 2024-09-17T22:57:46.182 INFO:teuthology.orchestra.run.smithi159.stdout:1647 write 0x5211093 thru 0x5219155 (0x80c3 bytes) 2024-09-17T22:57:46.184 INFO:teuthology.orchestra.run.smithi159.stdout:1648 writesame 0x29f7d1a thru 0x2a01d0d data_size 0x3a(0x9ff4 bytes) 2024-09-17T22:57:46.187 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:46.187 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:46.187 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:46.187 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:46.189 INFO:teuthology.orchestra.run.smithi159.stdout:1649 writesame 0xd7d0c40 thru 0xd7d4d6c data_size 0x5(0x412d bytes) 2024-09-17T22:57:46.192 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.193 INFO:teuthology.orchestra.run.smithi159.stdout:1651 writesame 0xc748565 thru 0xc756761 data_size 0x1(0xe1fd bytes) 2024-09-17T22:57:46.197 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:46.222 INFO:teuthology.orchestra.run.smithi159.stdout:1652 write 0x7dd55fb thru 0x7dd5940 (0x346 bytes) 2024-09-17T22:57:46.226 INFO:teuthology.orchestra.run.smithi159.stdout:1655 flatten 2024-09-17T22:57:46.277 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:46.275+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:46.297 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:46.297 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:46.301 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.301 INFO:teuthology.orchestra.run.smithi159.stdout:1657 writesame 0x676725d thru 0x6767a29 data_size 0x1(0x7cd bytes) 2024-09-17T22:57:46.304 INFO:teuthology.orchestra.run.smithi159.stdout:1658 write 0xc93e66c thru 0xc9439c2 (0x5357 bytes) 2024-09-17T22:57:46.312 INFO:teuthology.orchestra.run.smithi159.stdout:1659 trunc from 0xe062045 to 0xac8eaed 2024-09-17T22:57:46.315 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:46.315 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:46.325 INFO:teuthology.orchestra.run.smithi159.stdout:1662 trunc from 0xac8eaed to 0x8cbe973 2024-09-17T22:57:46.331 INFO:teuthology.orchestra.run.smithi159.stdout:1664 punch from 0x22a60a6 to 0x22a611c, (0x76 bytes) 2024-09-17T22:57:46.331 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.331 INFO:teuthology.orchestra.run.smithi159.stdout:1667 punch from 0x2d8f445 to 0x2d95379, (0x5f34 bytes) 2024-09-17T22:57:46.333 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:46.333 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:46.336 INFO:teuthology.orchestra.run.smithi159.stdout:1668 trunc from 0x8cbe973 to 0x4a657e3 2024-09-17T22:57:46.340 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:46.340 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:46.340 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:46.340 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:46.340 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:46.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:46.351 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:46.352 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:46.365 INFO:teuthology.orchestra.run.smithi159.stdout:1669 write 0x9bd041f thru 0x9bd64fd (0x60df bytes) 2024-09-17T22:57:46.368 INFO:teuthology.orchestra.run.smithi159.stdout:1671 read 0xcfeac6 thru 0xd0d1c2 (0xe6fd bytes) 2024-09-17T22:57:46.369 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:46.369 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:46.369 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:46.378 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:46.375+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:46.379 INFO:teuthology.orchestra.run.smithi159.stdout:1672 trunc from 0x9bd64fe to 0x253a1b0 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.c 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.1f 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.1c 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.17 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.16 2024-09-17T22:57:46.384 INFO:teuthology.orchestra.run.smithi171.stdout:3.d 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:3.11 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:3.5 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:3.1d 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:2.3 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:3.2 2024-09-17T22:57:46.385 INFO:teuthology.orchestra.run.smithi171.stdout:2.2 2024-09-17T22:57:46.392 INFO:teuthology.orchestra.run.smithi159.stdout:1674 read 0x12d5663 thru 0x12e0ff3 (0xb991 bytes) 2024-09-17T22:57:46.392 INFO:teuthology.orchestra.run.smithi159.stdout:1675 read 0x902329 thru 0x902b9c (0x874 bytes) 2024-09-17T22:57:46.393 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:46.393 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:46.406 INFO:teuthology.orchestra.run.smithi159.stdout:1681 write 0x9349d94 thru 0x93514bb (0x7728 bytes) 2024-09-17T22:57:46.412 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:46.412 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:46.415 INFO:teuthology.orchestra.run.smithi159.stdout:1682 write 0x9697b69 thru 0x9698250 (0x6e8 bytes) 2024-09-17T22:57:46.416 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:46.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:46.419 INFO:teuthology.orchestra.run.smithi159.stdout:1684 trunc from 0x9698251 to 0xa20b21c 2024-09-17T22:57:46.425 INFO:teuthology.orchestra.run.smithi159.stdout:1685 write 0x23377c7 thru 0x233bb34 (0x436e bytes) 2024-09-17T22:57:46.429 INFO:teuthology.orchestra.run.smithi159.stdout:1686 punch from 0x2976002 to 0x297b2f2, (0x52f0 bytes) 2024-09-17T22:57:46.429 INFO:teuthology.orchestra.run.smithi159.stdout:1687 writesame 0x540344d thru 0x5407ffa data_size 0x3(0x4bae bytes) 2024-09-17T22:57:46.433 INFO:teuthology.orchestra.run.smithi159.stdout:1690 write 0x3390c82 thru 0x3399d94 (0x9113 bytes) 2024-09-17T22:57:46.442 INFO:teuthology.orchestra.run.smithi159.stdout:1691 trunc from 0xa20b21c to 0x5deacb5 2024-09-17T22:57:46.455 INFO:teuthology.orchestra.run.smithi159.stdout:1693 read 0x579b1e7 thru 0x579ff61 (0x4d7b bytes) 2024-09-17T22:57:46.455 INFO:teuthology.orchestra.run.smithi159.stdout:1694 read 0xc69106 thru 0xc6b7de (0x26d9 bytes) 2024-09-17T22:57:46.455 INFO:teuthology.orchestra.run.smithi159.stdout:1695 read 0x29c1f24 thru 0x29c8259 (0x6336 bytes) 2024-09-17T22:57:46.455 INFO:teuthology.orchestra.run.smithi159.stdout:1697 punch from 0x5a1c77b to 0x5a26d21, (0xa5a6 bytes) 2024-09-17T22:57:46.457 INFO:teuthology.orchestra.run.smithi159.stdout:1698 trunc from 0x5deacb5 to 0x7a01cb1 2024-09-17T22:57:46.464 INFO:teuthology.orchestra.run.smithi159.stdout:1699 write 0x21bca46 thru 0x21c7a55 (0xb010 bytes) 2024-09-17T22:57:46.467 INFO:teuthology.orchestra.run.smithi159.stdout:1700 read 0x5ec0be1 thru 0x5ec344b (0x286b bytes) 2024-09-17T22:57:46.468 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:46.468 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:46.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:46.481 INFO:teuthology.orchestra.run.smithi159.stdout:1701 write 0xd820b53 thru 0xd82e16c (0xd61a bytes) 2024-09-17T22:57:46.485 INFO:teuthology.orchestra.run.smithi159.stdout:1702 writesame 0x6379fec thru 0x637facc data_size 0x63(0x5ae1 bytes) 2024-09-17T22:57:46.487 INFO:teuthology.orchestra.run.smithi159.stdout:1703 write 0x45a94bd thru 0x45ac7f6 (0x333a bytes) 2024-09-17T22:57:46.490 INFO:teuthology.orchestra.run.smithi159.stdout:1704 write 0x305dbe thru 0x30b540 (0x5783 bytes) 2024-09-17T22:57:46.492 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.492 INFO:teuthology.orchestra.run.smithi159.stdout:1707 writesame 0xae787f2 thru 0xae835eb data_size 0x27(0xadfa bytes) 2024-09-17T22:57:46.496 INFO:teuthology.orchestra.run.smithi159.stdout:1708 punch from 0x4731b51 to 0x4738389, (0x6838 bytes) 2024-09-17T22:57:46.504 INFO:teuthology.orchestra.run.smithi159.stdout:1709 trunc from 0xd82e16d to 0x68cdd52 2024-09-17T22:57:46.516 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.517 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.517 INFO:teuthology.orchestra.run.smithi159.stdout:1714 punch from 0x3e5e7d4 to 0x3e66f97, (0x87c3 bytes) 2024-09-17T22:57:46.519 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:46.516+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:46.524 INFO:teuthology.orchestra.run.smithi159.stdout:1715 trunc from 0x68cdd52 to 0xcff6da2 2024-09-17T22:57:46.530 INFO:teuthology.orchestra.run.smithi159.stdout:1716 write 0x3959aee thru 0x3969a04 (0xff17 bytes) 2024-09-17T22:57:46.533 INFO:teuthology.orchestra.run.smithi159.stdout:1718 write 0x2f0fd5f thru 0x2f1625e (0x6500 bytes) 2024-09-17T22:57:46.536 INFO:teuthology.orchestra.run.smithi159.stdout:1719 write 0x3eae3b1 thru 0x3eb00fd (0x1d4d bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1721 punch from 0x8144678 to 0x8150bde, (0xc566 bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1722 read 0x9164fc1 thru 0x9172231 (0xd271 bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1723 read 0x5711e02 thru 0x5715b40 (0x3d3f bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1724 punch from 0x2b361db to 0x2b37ad8, (0x18fd bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1725 read 0x8c9ce5a thru 0x8ca627c (0x9423 bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1726 punch from 0x68a0e83 to 0x68a769d, (0x681a bytes) 2024-09-17T22:57:46.540 INFO:teuthology.orchestra.run.smithi159.stdout:1727 read 0x80b5c7e thru 0x80c11b6 (0xb539 bytes) 2024-09-17T22:57:46.541 INFO:teuthology.orchestra.run.smithi159.stdout:1728 read 0x8fa2d4f thru 0x8faf8c4 (0xcb76 bytes) 2024-09-17T22:57:46.541 INFO:teuthology.orchestra.run.smithi159.stdout:1729 read 0x7f57f4a thru 0x7f61ee2 (0x9f99 bytes) 2024-09-17T22:57:46.541 INFO:teuthology.orchestra.run.smithi159.stdout:1730 punch from 0x4799e1 to 0x47c1ca, (0x27e9 bytes) 2024-09-17T22:57:46.541 INFO:teuthology.orchestra.run.smithi159.stdout:1731 writesame 0x6de01ad thru 0x6de56e6 data_size 0x1(0x553a bytes) 2024-09-17T22:57:46.552 INFO:teuthology.orchestra.run.smithi159.stdout:1732 read 0x4ea0628 thru 0x4eab9eb (0xb3c4 bytes) 2024-09-17T22:57:46.552 INFO:teuthology.orchestra.run.smithi159.stdout:1733 read 0x4320c00 thru 0x4326b43 (0x5f44 bytes) 2024-09-17T22:57:46.553 INFO:teuthology.orchestra.run.smithi159.stdout:1734 punch from 0xa746299 to 0xa753e88, (0xdbef bytes) 2024-09-17T22:57:46.553 INFO:teuthology.orchestra.run.smithi159.stdout:1735 read 0x69bdde8 thru 0x69cd8fa (0xfb13 bytes) 2024-09-17T22:57:46.553 INFO:teuthology.orchestra.run.smithi159.stdout:1736 write 0x9eb6aa3 thru 0x9ebec7f (0x81dd bytes) 2024-09-17T22:57:46.555 INFO:teuthology.orchestra.run.smithi159.stdout:1737 write 0xa23b18f thru 0xa247e53 (0xccc5 bytes) 2024-09-17T22:57:46.570 INFO:teuthology.orchestra.run.smithi159.stdout:1738 trunc from 0xcff6da2 to 0x2acd74b 2024-09-17T22:57:46.580 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:46.578+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:46.591 INFO:teuthology.orchestra.run.smithi159.stdout:1739 write 0x63f2fdc thru 0x63f6916 (0x393b bytes) 2024-09-17T22:57:46.593 INFO:teuthology.orchestra.run.smithi159.stdout:1740 read 0x1d0d517 thru 0x1d17bb1 (0xa69b bytes) 2024-09-17T22:57:46.596 INFO:teuthology.orchestra.run.smithi159.stdout:1742 trunc from 0x63f6917 to 0x8194dcd 2024-09-17T22:57:46.601 INFO:teuthology.orchestra.run.smithi159.stdout:1743 punch from 0x2989059 to 0x2997bba, (0xeb61 bytes) 2024-09-17T22:57:46.601 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.602 INFO:teuthology.orchestra.run.smithi159.stdout:1746 punch from 0x78a021d to 0x78a86ed, (0x84d0 bytes) 2024-09-17T22:57:46.602 INFO:teuthology.orchestra.run.smithi159.stdout:1747 read 0x178651a thru 0x1791976 (0xb45d bytes) 2024-09-17T22:57:46.603 INFO:teuthology.orchestra.run.smithi159.stdout:1748 trunc from 0x8194dcd to 0x8d50090 2024-09-17T22:57:46.619 INFO:teuthology.orchestra.run.smithi159.stdout:1749 writesame 0xddc84fe thru 0xddceb2b data_size 0x3a(0x662e bytes) 2024-09-17T22:57:46.619 INFO:teuthology.orchestra.run.smithi159.stdout:1750 writesame 0x26199b8 thru 0x261de6f data_size 0xc(0x44b8 bytes) 2024-09-17T22:57:46.623 INFO:teuthology.orchestra.run.smithi159.stdout:1751 punch from 0x1c41b1a to 0x1c4fd12, (0xe1f8 bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1752 read 0x39a1069 thru 0x39a2951 (0x18e9 bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1753 read 0x733e0d6 thru 0x73477e7 (0x9712 bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1755 read 0x7f58724 thru 0x7f5dcf0 (0x55cd bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1759 read 0x4feefe8 thru 0x4ffc19f (0xd1b8 bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1760 punch from 0x4509bcd to 0x4512197, (0x85ca bytes) 2024-09-17T22:57:46.624 INFO:teuthology.orchestra.run.smithi159.stdout:1761 writesame 0x337f49f thru 0x338bce7 data_size 0x51(0xc849 bytes) 2024-09-17T22:57:46.628 INFO:teuthology.orchestra.run.smithi159.stdout:1762 writesame 0x5b36fc0 thru 0x5b3d478 data_size 0x2d(0x64b9 bytes) 2024-09-17T22:57:46.630 INFO:teuthology.orchestra.run.smithi159.stdout:1763 read 0x428453c thru 0x4291bb4 (0xd679 bytes) 2024-09-17T22:57:46.633 INFO:teuthology.orchestra.run.smithi159.stdout:1764 trunc from 0xddceb2c to 0xc410415 2024-09-17T22:57:46.634 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:46.634 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:46.640 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.640 INFO:teuthology.orchestra.run.smithi159.stdout:1768 read 0x43e22ff thru 0x43ef828 (0xd52a bytes) 2024-09-17T22:57:46.640 INFO:teuthology.orchestra.run.smithi159.stdout:1770 write 0xb564fd2 thru 0xb569e87 (0x4eb6 bytes) 2024-09-17T22:57:46.643 INFO:teuthology.orchestra.run.smithi159.stdout:1771 read 0x7828823 thru 0x7834c8c (0xc46a bytes) 2024-09-17T22:57:46.643 INFO:teuthology.orchestra.run.smithi159.stdout:1772 read 0x8d32c13 thru 0x8d3b15e (0x854c bytes) 2024-09-17T22:57:46.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:46.652 INFO:teuthology.orchestra.run.smithi159.stdout:1773 write 0xedeaa73 thru 0xedee171 (0x36ff bytes) 2024-09-17T22:57:46.655 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.655 INFO:teuthology.orchestra.run.smithi159.stdout:1776 write 0x9a42fa thru 0x9aeb82 (0xa889 bytes) 2024-09-17T22:57:46.657 INFO:teuthology.orchestra.run.smithi159.stdout:1777 read 0xc27cd9 thru 0xc2a328 (0x2650 bytes) 2024-09-17T22:57:46.657 INFO:teuthology.orchestra.run.smithi159.stdout:1779 write 0xd9b18da thru 0xd9b3fab (0x26d2 bytes) 2024-09-17T22:57:46.659 INFO:teuthology.orchestra.run.smithi159.stdout:1780 write 0xd30ec0f thru 0xd31d8fb (0xeced bytes) 2024-09-17T22:57:46.663 INFO:teuthology.orchestra.run.smithi159.stdout:1782 punch from 0x2bc19b3 to 0x2bc8a4a, (0x7097 bytes) 2024-09-17T22:57:46.663 INFO:teuthology.orchestra.run.smithi159.stdout:1783 punch from 0xdde70de to 0xddec289, (0x51ab bytes) 2024-09-17T22:57:46.663 INFO:teuthology.orchestra.run.smithi159.stdout:1784 write 0x264698e thru 0x264d9c0 (0x7033 bytes) 2024-09-17T22:57:46.666 INFO:teuthology.orchestra.run.smithi159.stdout:1785 punch from 0xed24a36 to 0xed2ce83, (0x844d bytes) 2024-09-17T22:57:46.666 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.667 INFO:teuthology.orchestra.run.smithi159.stdout:1787 write 0xb40b6bd thru 0xb41342d (0x7d71 bytes) 2024-09-17T22:57:46.667 INFO:teuthology.orchestra.run.smithi159.stdout:1788 writesame 0xddbab5c thru 0xddc7baf data_size 0x86(0xd054 bytes) 2024-09-17T22:57:46.671 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.671 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.671 INFO:teuthology.orchestra.run.smithi159.stdout:1793 write 0x56c34bb thru 0x56c6373 (0x2eb9 bytes) 2024-09-17T22:57:46.675 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:46.679 INFO:teuthology.orchestra.run.smithi159.stdout:1796 trunc from 0xedee172 to 0xb45d938 2024-09-17T22:57:46.687 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:46.684+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:46.692 INFO:teuthology.orchestra.run.smithi159.stdout:1798 clone 16 order 24 su 524288 sc 15 2024-09-17T22:57:46.750 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:46.751 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:46.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:46.782 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:46.780+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:46.883 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:46.880+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:46.904 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:46.904 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:46.912 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.16 --file /var/log/ceph/exp.3.16.5 2024-09-17T22:57:46.913 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:46.984 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:46.982+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:47.019 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.019 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.027 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:47.111 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:47.109+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:47.179 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.180 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.186 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:47.183+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:47.189 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:47.272 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T22:57:47.286 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:47.283+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:47.303 INFO:teuthology.orchestra.run.smithi159.stdout:1799 read 0xa6d713b thru 0xa6e4431 (0xd2f7 bytes) 2024-09-17T22:57:47.315 INFO:teuthology.orchestra.run.smithi159.stdout:1801 trunc from 0xb45d938 to 0x16b1675 2024-09-17T22:57:47.347 INFO:teuthology.orchestra.run.smithi159.stdout:1803 read 0x1259ab8 thru 0x1263c75 (0xa1be bytes) 2024-09-17T22:57:47.356 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.356 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.369 INFO:teuthology.orchestra.run.smithi159.stdout:1805 write 0xee15809 thru 0xee21654 (0xbe4c bytes) 2024-09-17T22:57:47.370 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:47.371 INFO:teuthology.orchestra.run.smithi159.stdout:1806 read 0xc9c95b5 thru 0xc9d0308 (0x6d54 bytes) 2024-09-17T22:57:47.371 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.371 INFO:teuthology.orchestra.run.smithi159.stdout:1808 read 0x12c4353 thru 0x12c89fe (0x46ac bytes) 2024-09-17T22:57:47.372 INFO:teuthology.orchestra.run.smithi159.stdout:1809 write 0x170a8a thru 0x1713f5 (0x96c bytes) 2024-09-17T22:57:47.373 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.375 INFO:teuthology.orchestra.run.smithi159.stdout:1812 trunc from 0xee21655 to 0xd1ffb10 2024-09-17T22:57:47.387 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:47.384+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:47.394 INFO:teuthology.orchestra.run.smithi159.stdout:1813 write 0xe7b4b71 thru 0xe7bca28 (0x7eb8 bytes) 2024-09-17T22:57:47.395 INFO:teuthology.orchestra.run.smithi159.stdout:1814 read 0x25a45e0 thru 0x25a8fc3 (0x49e4 bytes) 2024-09-17T22:57:47.396 INFO:teuthology.orchestra.run.smithi159.stdout:1815 punch from 0x174f4f4 to 0x1752198, (0x2ca4 bytes) 2024-09-17T22:57:47.396 INFO:teuthology.orchestra.run.smithi159.stdout:1816 read 0xccad58a thru 0xccbc9b7 (0xf42e bytes) 2024-09-17T22:57:47.397 INFO:teuthology.orchestra.run.smithi159.stdout:1818 punch from 0x520cbae to 0x521c6c5, (0xfb17 bytes) 2024-09-17T22:57:47.397 INFO:teuthology.orchestra.run.smithi159.stdout:1819 read 0xa05c0b9 thru 0xa069cce (0xdc16 bytes) 2024-09-17T22:57:47.397 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.397 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.397 INFO:teuthology.orchestra.run.smithi159.stdout:1823 writesame 0x2cd7d13 thru 0x2cdbb6f data_size 0x5(0x3e5d bytes) 2024-09-17T22:57:47.399 INFO:teuthology.orchestra.run.smithi159.stdout:1825 punch from 0xaeae4b8 to 0xaebdce8, (0xf830 bytes) 2024-09-17T22:57:47.399 INFO:teuthology.orchestra.run.smithi159.stdout:1826 read 0x75d7cfd thru 0x75dd88f (0x5b93 bytes) 2024-09-17T22:57:47.411 INFO:teuthology.orchestra.run.smithi159.stdout:1827 trunc from 0xe7bca29 to 0x4da88b7 2024-09-17T22:57:47.420 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.438 INFO:teuthology.orchestra.run.smithi159.stdout:1830 write 0xed67309 thru 0xed758cf (0xe5c7 bytes) 2024-09-17T22:57:47.439 INFO:teuthology.orchestra.run.smithi159.stdout:1832 write 0x16ea737 thru 0x16f74e2 (0xcdac bytes) 2024-09-17T22:57:47.440 INFO:teuthology.orchestra.run.smithi159.stdout:1833 punch from 0x7cdffa6 to 0x7ce6234, (0x628e bytes) 2024-09-17T22:57:47.440 INFO:teuthology.orchestra.run.smithi159.stdout:1834 write 0xa6a6630 thru 0xa6b3019 (0xc9ea bytes) 2024-09-17T22:57:47.441 INFO:teuthology.orchestra.run.smithi159.stdout:1835 writesame 0x88bd03d thru 0x88ccf18 data_size 0x6(0xfedc bytes) 2024-09-17T22:57:47.446 INFO:teuthology.orchestra.run.smithi159.stdout:1836 read 0x5e5a2f9 thru 0x5e5d629 (0x3331 bytes) 2024-09-17T22:57:47.446 INFO:teuthology.orchestra.run.smithi159.stdout:1837 write 0x4f739a3 thru 0x4f782a4 (0x4902 bytes) 2024-09-17T22:57:47.448 INFO:teuthology.orchestra.run.smithi159.stdout:1838 writesame 0x20e4bed thru 0x20ed455 data_size 0x1(0x8869 bytes) 2024-09-17T22:57:47.466 INFO:teuthology.orchestra.run.smithi159.stdout:1839 read 0xb0bd1e4 thru 0xb0c5b57 (0x8974 bytes) 2024-09-17T22:57:47.481 INFO:teuthology.orchestra.run.smithi159.stdout:1840 trunc from 0xed758d0 to 0x22d61dc 2024-09-17T22:57:47.488 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:47.485+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:47.489 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.489 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.492 INFO:teuthology.orchestra.run.smithi159.stdout:1842 trunc from 0x22d61dc to 0x1c0de0b 2024-09-17T22:57:47.500 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:47.519 INFO:teuthology.orchestra.run.smithi159.stdout:1843 writesame 0xa87315c thru 0xa880cbc data_size 0x47(0xdb61 bytes) 2024-09-17T22:57:47.523 INFO:teuthology.orchestra.run.smithi159.stdout:1845 trunc from 0xa880cbd to 0xd68a04d 2024-09-17T22:57:47.536 INFO:teuthology.orchestra.run.smithi159.stdout:1846 trunc from 0xd68a04d to 0x69bb628 2024-09-17T22:57:47.545 INFO:teuthology.orchestra.run.smithi159.stdout:1847 write 0x3b8390a thru 0x3b8c20e (0x8905 bytes) 2024-09-17T22:57:47.551 INFO:teuthology.orchestra.run.smithi159.stdout:1849 trunc from 0x69bb628 to 0xa68991a 2024-09-17T22:57:47.560 INFO:teuthology.orchestra.run.smithi159.stdout:1850 write 0xa9eb6dd thru 0xa9ee1fb (0x2b1f bytes) 2024-09-17T22:57:47.562 INFO:teuthology.orchestra.run.smithi159.stdout:1851 punch from 0x9cc73ad to 0x9cd2546, (0xb199 bytes) 2024-09-17T22:57:47.562 INFO:teuthology.orchestra.run.smithi159.stdout:1852 writesame 0x298d374 thru 0x2998b2d data_size 0x4e(0xb7ba bytes) 2024-09-17T22:57:47.570 INFO:teuthology.orchestra.run.smithi159.stdout:1853 write 0xbf04c08 thru 0xbf053cb (0x7c4 bytes) 2024-09-17T22:57:47.571 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.571 INFO:teuthology.orchestra.run.smithi159.stdout:1855 read 0x1a621bd thru 0x1a678fa (0x573e bytes) 2024-09-17T22:57:47.572 INFO:teuthology.orchestra.run.smithi159.stdout:1856 write 0x42ba9b4 thru 0x42bb3c9 (0xa16 bytes) 2024-09-17T22:57:47.573 INFO:teuthology.orchestra.run.smithi159.stdout:1857 writesame 0x1c5e4ee thru 0x1c6a1fc data_size 0x49(0xbd0f bytes) 2024-09-17T22:57:47.575 INFO:teuthology.orchestra.run.smithi159.stdout:1859 write 0x30292fb thru 0x3032241 (0x8f47 bytes) 2024-09-17T22:57:47.576 INFO:teuthology.orchestra.run.smithi159.stdout:1860 writesame 0x9fccfca thru 0x9fd3dbd data_size 0x3e(0x6df4 bytes) 2024-09-17T22:57:47.584 INFO:teuthology.orchestra.run.smithi159.stdout:1861 trunc from 0xbf053cc to 0x61ffd28 2024-09-17T22:57:47.589 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:47.585+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:47.597 INFO:teuthology.orchestra.run.smithi159.stdout:1863 read 0x20f4bde thru 0x20f9337 (0x475a bytes) 2024-09-17T22:57:47.597 INFO:teuthology.orchestra.run.smithi159.stdout:1865 read 0x333b105 thru 0x334913b (0xe037 bytes) 2024-09-17T22:57:47.597 INFO:teuthology.orchestra.run.smithi159.stdout:1867 read 0x2b0dd7 thru 0x2bd098 (0xc2c2 bytes) 2024-09-17T22:57:47.603 INFO:teuthology.orchestra.run.smithi159.stdout:1868 trunc from 0x61ffd28 to 0x9f85890 2024-09-17T22:57:47.609 INFO:teuthology.orchestra.run.smithi159.stdout:1870 write 0x10da081 thru 0x10e083e (0x67be bytes) 2024-09-17T22:57:47.611 INFO:teuthology.orchestra.run.smithi159.stdout:1871 write 0x20f53ef thru 0x2102b86 (0xd798 bytes) 2024-09-17T22:57:47.614 INFO:teuthology.orchestra.run.smithi159.stdout:1873 flatten 2024-09-17T22:57:47.618 INFO:teuthology.orchestra.run.smithi159.stdout:1874 write 0x9beb93d thru 0x9bfa022 (0xe6e6 bytes) 2024-09-17T22:57:47.623 INFO:teuthology.orchestra.run.smithi159.stdout:1875 trunc from 0x9f85890 to 0x7afee9c 2024-09-17T22:57:47.637 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.637 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.639 INFO:teuthology.orchestra.run.smithi159.stdout:1876 write 0x863c4c0 thru 0x8648a92 (0xc5d3 bytes) 2024-09-17T22:57:47.640 INFO:teuthology.orchestra.run.smithi159.stdout:1877 trunc from 0x8648a93 to 0x89c5237 2024-09-17T22:57:47.645 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.645 INFO:teuthology.orchestra.run.smithi159.stdout:1880 write 0x2d9927a thru 0x2da8f33 (0xfcba bytes) 2024-09-17T22:57:47.647 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:47.659 INFO:teuthology.orchestra.run.smithi159.stdout:1881 write 0xd62309c thru 0xd62e188 (0xb0ed bytes) 2024-09-17T22:57:47.661 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.661 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:47.661 INFO:teuthology.orchestra.run.smithi159.stdout:1884 read 0xa57bef7 thru 0xa5878bd (0xb9c7 bytes) 2024-09-17T22:57:47.676 INFO:teuthology.orchestra.run.smithi159.stdout:1885 trunc from 0xd62e189 to 0x114b4b0 2024-09-17T22:57:47.684 INFO:teuthology.orchestra.run.smithi159.stdout:1886 clone 17 order 23 su 131072 sc 3 2024-09-17T22:57:47.690 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:47.688+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:47.767 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.767 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.776 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:47.817 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:47.814+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:47.892 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:47.890+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:47.918 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:47.918 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:47.928 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:47.993 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:47.990+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:48.031 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:48.031 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:48.039 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:48.094 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:48.091+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:48.095 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:57:48.151 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:48.151 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:48.161 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:48.182 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T22:57:48.195 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:48.193+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:48.297 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:48.293+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:48.303 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:48.304 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:48.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:48.353 INFO:teuthology.orchestra.run.smithi171.stderr:Exporting 3.16 info 3.16( v 45'87 (0'0,45'87] local-lis/les=22/23 n=16 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-09-17T22:57:48.360 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6834e080:::rbd_data.1188373e3d04.000000000000000d:6# 2024-09-17T22:57:48.362 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6834e080:::rbd_data.1188373e3d04.000000000000000d:head# 2024-09-17T22:57:48.362 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:68450ac3:::rbd_data.118834c4f370.0000000000000064:head# 2024-09-17T22:57:48.362 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:68dcf43e:::rbd_data.118834c4f370.000000000000004a:head# 2024-09-17T22:57:48.362 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:69191c60:::rbd_data.118846a2cbc0.000000000000000d:9# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:69191c60:::rbd_data.118846a2cbc0.000000000000000d:head# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:69bce82f:::rbd_data.1188903913c1.000000000000002c:11# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:69bce82f:::rbd_data.1188903913c1.000000000000002c:head# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6a416ce2:::rbd_object_map.1188f4a20928.0000000000000004:head# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6a4cb7f8:::rbd_data.118840f89003.00000000000000bb:head# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6ad3dd83:::rbd_data.118840f89003.00000000000000d4:head# 2024-09-17T22:57:48.364 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6be9c72b:::rbd_data.118840f89003.00000000000000da:head# 2024-09-17T22:57:48.365 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6bf17a9c:::rbd_data.118846a2cbc0.000000000000000e:9# 2024-09-17T22:57:48.365 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6bf17a9c:::rbd_data.118846a2cbc0.000000000000000e:head# 2024-09-17T22:57:48.365 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6de7e899:::rbd_data.118840f89003.00000000000000d7:head# 2024-09-17T22:57:48.366 INFO:teuthology.orchestra.run.smithi171.stderr:Read #3:6e770d7a:::rbd_data.1188f4a20928.0000000000000002:head# 2024-09-17T22:57:48.373 INFO:teuthology.orchestra.run.smithi171.stderr:Export successful 2024-09-17T22:57:48.392 INFO:teuthology.orchestra.run.smithi159.stdout:1888 read 0xb951ed thru 0xba39c2 (0xe7d6 bytes) 2024-09-17T22:57:48.397 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:48.395+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:48.403 INFO:teuthology.orchestra.run.smithi159.stdout:1889 write 0x21c0931 thru 0x21ce0c7 (0xd797 bytes) 2024-09-17T22:57:48.413 INFO:teuthology.orchestra.run.smithi159.stdout:1893 write 0x3787cc5 thru 0x37926b1 (0xa9ed bytes) 2024-09-17T22:57:48.415 INFO:teuthology.orchestra.run.smithi159.stdout:1894 punch from 0x30975d0 to 0x3098308, (0xd38 bytes) 2024-09-17T22:57:48.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:48.433 INFO:teuthology.orchestra.run.smithi159.stdout:1895 writesame 0xdedd5ed thru 0xdedfe5a data_size 0x4b(0x286e bytes) 2024-09-17T22:57:48.433 INFO:teuthology.orchestra.run.smithi159.stdout:1896 write 0x8848474 thru 0x8849d85 (0x1912 bytes) 2024-09-17T22:57:48.437 INFO:teuthology.orchestra.run.smithi159.stdout:1901 flatten 2024-09-17T22:57:48.461 INFO:teuthology.orchestra.run.smithi159.stdout:1902 read 0xd57bf63 thru 0xd580b35 (0x4bd3 bytes) 2024-09-17T22:57:48.461 INFO:teuthology.orchestra.run.smithi159.stdout:1903 write 0x5001901 thru 0x5005fd9 (0x46d9 bytes) 2024-09-17T22:57:48.470 INFO:teuthology.orchestra.run.smithi159.stdout:1904 writesame 0xe390d59 thru 0xe39d5b0 data_size 0xc(0xc858 bytes) 2024-09-17T22:57:48.471 INFO:teuthology.orchestra.run.smithi159.stdout:1906 write 0xad69cf9 thru 0xad760a5 (0xc3ad bytes) 2024-09-17T22:57:48.474 INFO:teuthology.orchestra.run.smithi159.stdout:1907 read 0xb299507 thru 0xb2a8f69 (0xfa63 bytes) 2024-09-17T22:57:48.474 INFO:teuthology.orchestra.run.smithi159.stdout:1908 read 0xe1d89cc thru 0xe1e6919 (0xdf4e bytes) 2024-09-17T22:57:48.474 INFO:teuthology.orchestra.run.smithi159.stdout:1909 writesame 0x214face thru 0x2151c27 data_size 0x3(0x215a bytes) 2024-09-17T22:57:48.477 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:48.478 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:48.478 INFO:teuthology.orchestra.run.smithi159.stdout:1910 read 0x94d6730 thru 0x94e3d19 (0xd5ea bytes) 2024-09-17T22:57:48.479 INFO:teuthology.orchestra.run.smithi159.stdout:1911 read 0x6a7ebc6 thru 0x6a812c4 (0x26ff bytes) 2024-09-17T22:57:48.480 INFO:teuthology.orchestra.run.smithi159.stdout:1913 writesame 0xc2bdff2 thru 0xc2c06f4 data_size 0x1(0x2703 bytes) 2024-09-17T22:57:48.485 INFO:teuthology.orchestra.run.smithi159.stdout:1914 writesame 0x45c7c9c thru 0x45d0e96 data_size 0x1(0x91fb bytes) 2024-09-17T22:57:48.488 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:48.499 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:48.495+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:48.509 INFO:teuthology.orchestra.run.smithi159.stdout:1915 trunc from 0xe39d5b1 to 0x697580f 2024-09-17T22:57:48.519 INFO:teuthology.orchestra.run.smithi159.stdout:1916 read 0x31af758 thru 0x31b5bb0 (0x6459 bytes) 2024-09-17T22:57:48.519 INFO:teuthology.orchestra.run.smithi159.stdout:1917 read 0x41cee83 thru 0x41d8d6e (0x9eec bytes) 2024-09-17T22:57:48.519 INFO:teuthology.orchestra.run.smithi159.stdout:1918 punch from 0x4b9c500 to 0x4b9e647, (0x2147 bytes) 2024-09-17T22:57:48.519 INFO:teuthology.orchestra.run.smithi159.stdout:1919 punch from 0x3339a74 to 0x33486b5, (0xec41 bytes) 2024-09-17T22:57:48.520 INFO:teuthology.orchestra.run.smithi159.stdout:1920 trunc from 0x697580f to 0x55c27e9 2024-09-17T22:57:48.535 INFO:teuthology.orchestra.run.smithi159.stdout:1923 writesame 0x7bb5ec6 thru 0x7bbb049 data_size 0xc(0x5184 bytes) 2024-09-17T22:57:48.535 INFO:teuthology.orchestra.run.smithi159.stdout:1924 read 0x459d2da thru 0x45a29f4 (0x571b bytes) 2024-09-17T22:57:48.535 INFO:teuthology.orchestra.run.smithi159.stdout:1925 writesame 0x3a1f137 thru 0x3a21b09 data_size 0x3(0x29d3 bytes) 2024-09-17T22:57:48.539 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.540 INFO:teuthology.orchestra.run.smithi159.stdout:1927 writesame 0x497b2a0 thru 0x4983e1b data_size 0x71(0x8b7c bytes) 2024-09-17T22:57:48.544 INFO:teuthology.orchestra.run.smithi159.stdout:1928 writesame 0x69bb467 thru 0x69c06e0 data_size 0x45(0x527a bytes) 2024-09-17T22:57:48.548 INFO:teuthology.orchestra.run.smithi159.stdout:1929 write 0x621737a thru 0x6221893 (0xa51a bytes) 2024-09-17T22:57:48.550 INFO:teuthology.orchestra.run.smithi159.stdout:1930 read 0x27d3e14 thru 0x27d86e2 (0x48cf bytes) 2024-09-17T22:57:48.551 INFO:teuthology.orchestra.run.smithi159.stdout:1932 trunc from 0x7bbb04a to 0x86e022f 2024-09-17T22:57:48.557 INFO:teuthology.orchestra.run.smithi159.stdout:1933 punch from 0x4263bb6 to 0x426eb22, (0xaf6c bytes) 2024-09-17T22:57:48.557 INFO:teuthology.orchestra.run.smithi159.stdout:1934 read 0x4eef762 thru 0x4efbfa1 (0xc840 bytes) 2024-09-17T22:57:48.557 INFO:teuthology.orchestra.run.smithi159.stdout:1935 writesame 0x121fc01 thru 0x12273af data_size 0x7(0x77af bytes) 2024-09-17T22:57:48.560 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.560 INFO:teuthology.orchestra.run.smithi159.stdout:1937 write 0x5c22eb7 thru 0x5c30d9d (0xdee7 bytes) 2024-09-17T22:57:48.561 INFO:teuthology.orchestra.run.smithi159.stdout:1938 read 0xbff994 thru 0xc005ae (0xc1b bytes) 2024-09-17T22:57:48.563 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:48.565 INFO:teuthology.orchestra.run.smithi159.stdout:1939 trunc from 0x86e022f to 0xad6ea5c 2024-09-17T22:57:48.579 INFO:teuthology.orchestra.run.smithi159.stdout:1941 write 0xd9853e7 thru 0xd9854e2 (0xfc bytes) 2024-09-17T22:57:48.581 INFO:teuthology.orchestra.run.smithi159.stdout:1942 punch from 0xd57e05a to 0xd58b9cb, (0xd971 bytes) 2024-09-17T22:57:48.581 INFO:teuthology.orchestra.run.smithi159.stdout:1943 write 0x7f7d645 thru 0x7f8d3ba (0xfd76 bytes) 2024-09-17T22:57:48.583 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:48.583 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:48.585 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.588 INFO:teuthology.orchestra.run.smithi159.stdout:1945 trunc from 0xd9854e3 to 0xb6c1d13 2024-09-17T22:57:48.598 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.598 INFO:teuthology.orchestra.run.smithi159.stdout:1947 write 0x4beb95f thru 0x4bec503 (0xba5 bytes) 2024-09-17T22:57:48.599 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:48.596+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:48.599 INFO:teuthology.orchestra.run.smithi159.stdout:1948 punch from 0x5444a2d to 0x54470db, (0x26ae bytes) 2024-09-17T22:57:48.602 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:48.603 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:48.612 INFO:teuthology.orchestra.run.smithi159.stdout:1949 trunc from 0xb6c1d13 to 0xfcbf98 2024-09-17T22:57:48.622 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:48.625 INFO:teuthology.orchestra.run.smithi159.stdout:1950 read 0xd5dbe2 thru 0xd61e26 (0x4245 bytes) 2024-09-17T22:57:48.636 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:48.636 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:48.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5757 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045330721999999, 2024-09-17T22:57:48.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038199444999999999, 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.638 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.639 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5757 2024-09-17T22:57:48.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.639 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.640 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:48.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5600000000000002e-07 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586464+0000", 2024-09-17T22:57:48.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586468+0000", 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8630000000000001e-06 2024-09-17T22:57:48.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586471+0000", 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7549999999999999e-06 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624560+0000", 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038088960999999998 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624607+0000", 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6740000000000003e-05 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624661+0000", 2024-09-17T22:57:48.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4732999999999999e-05 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.647 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5758 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045317832999999, 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038315071999999999, 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.648 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.649 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5758 2024-09-17T22:57:48.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5100000000000001e-07 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586476+0000", 2024-09-17T22:57:48.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.113e-06 2024-09-17T22:57:48.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586483+0000", 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5200000000000003e-06 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586487+0000", 2024-09-17T22:57:48.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3479999999999997e-06 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624681+0000", 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038194371999999997 2024-09-17T22:57:48.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624722+0000", 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0367999999999997e-05 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624790+0000", 2024-09-17T22:57:48.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8100000000000002e-05 2024-09-17T22:57:48.657 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.657 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5759 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3948544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045302108000001, 2024-09-17T22:57:48.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038450864000000001, 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5759 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:48.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586491+0000", 2024-09-17T22:57:48.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586492+0000", 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5349999999999999e-06 2024-09-17T22:57:48.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586495+0000", 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586498+0000", 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6170000000000001e-06 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624828+0000", 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038329514000000002 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624875+0000", 2024-09-17T22:57:48.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7107999999999997e-05 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624941+0000", 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6647000000000001e-05 2024-09-17T22:57:48.667 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.668 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.668 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.668 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5760 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045291401, 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038669935000000003, 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.669 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5760 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.670 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586503+0000", 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0899999999999999e-06 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586506+0000", 2024-09-17T22:57:48.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586508+0000", 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624980+0000", 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038472343999999999 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625076+0000", 2024-09-17T22:57:48.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5214e-05 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625171+0000", 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5421999999999995e-05 2024-09-17T22:57:48.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5761 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4276224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:48.678 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045281420999999, 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038805352000000001, 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5761 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.22e-07 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586512+0000", 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0210000000000001e-06 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586515+0000", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586518+0000", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625211+0000", 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038693268000000003 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625261+0000", 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9345000000000003e-05 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625317+0000", 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5850999999999998e-05 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5762 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4472832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045271530000001, 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038916981000000003, 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5762 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:57:48.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586525+0000", 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.224e-06 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586532+0000", 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3950000000000003e-06 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586535+0000", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625339+0000", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038804584000000003 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625385+0000", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5099000000000003e-05 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625438+0000", 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3511999999999999e-05 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5763 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4669440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045253599999999, 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039029253999999999, 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.688 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5763 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586541+0000", 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.268e-06 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586547+0000", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586554+0000", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625465+0000", 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038910924 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.691 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625511+0000", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5179999999999998e-05 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625568+0000", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7738999999999997e-05 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5764 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4866048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:48.692 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045232749, 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039122912000000003, 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5764 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586561+0000", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.266e-06 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586566+0000", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586568+0000", 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625592+0000", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039024174000000002 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625634+0000", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1464000000000001e-05 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625683+0000", 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9002000000000003e-05 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.696 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5765 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5062656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.045220902000001, 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039240865999999999, 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5765 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586895+0000", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000323658 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586899+0000", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6260000000000001e-06 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586910+0000", 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1239e-05 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586917+0000", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625710+0000", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038792608999999999 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625757+0000", 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7416000000000001e-05 2024-09-17T22:57:48.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625813+0000", 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5395000000000001e-05 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5766 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5259264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.04486477, 2024-09-17T22:57:48.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038992026999999999, 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5766 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:48.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586931+0000", 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586939+0000", 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2549999999999997e-06 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586945+0000", 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625828+0000", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038882947000000001 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625871+0000", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3204000000000002e-05 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625920+0000", 2024-09-17T22:57:48.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8539999999999999e-05 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5767 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5455872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.044839679999999, 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039097260000000002, 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.706 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5767 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588020+0000", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00106675 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588024+0000", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588033+0000", 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588044+0000", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1800000000000001e-05 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625943+0000", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037898695000000003 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625988+0000", 2024-09-17T22:57:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4577999999999999e-05 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626050+0000", 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2605000000000001e-05 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5769 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5849088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.043709048, 2024-09-17T22:57:48.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038207729000000003, 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5769 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:48.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588086+0000", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.103e-06 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588094+0000", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6459999999999993e-06 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588099+0000", 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1259999999999997e-06 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626190+0000", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.03809067 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626234+0000", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4051999999999998e-05 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626291+0000", 2024-09-17T22:57:48.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7694e-05 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5770 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.043686076, 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038303865999999999, 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.715 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5770 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:48.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588109+0000", 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0899999999999999e-06 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588116+0000", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2439999999999996e-06 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588122+0000", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8640000000000001e-06 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626315+0000", 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038193105999999998 2024-09-17T22:57:48.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626359+0000", 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3828000000000001e-05 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626410+0000", 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1328000000000001e-05 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.719 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5771 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.043663796000001, 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038402700999999997, 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5771 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588131+0000", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588138+0000", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3049999999999998e-06 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588143+0000", 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4650000000000002e-06 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626433+0000", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038289895999999997 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626476+0000", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2913000000000001e-05 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626532+0000", 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5692000000000001e-05 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.723 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5772 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6438912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.043642137999999, 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038494234000000002, 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5772 2024-09-17T22:57:48.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590017+0000", 2024-09-17T22:57:48.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018664980000000001 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590020+0000", 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4769999999999999e-06 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590027+0000", 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5209999999999999e-06 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590032+0000", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626554+0000", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036521551999999999 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626595+0000", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1900000000000002e-05 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626645+0000", 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9205000000000001e-05 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5828 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.041448730999999, 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038160161999999997, 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5828 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4600000000000001e-07 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590345+0000", 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.376e-06 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590349+0000", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590352+0000", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7410000000000001e-06 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628400+0000", 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038048347000000003 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628448+0000", 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7961999999999999e-05 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628504+0000", 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6137000000000001e-05 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5829 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.041437676000001, 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038295726000000002, 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5829 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.733 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592028+0000", 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001673379 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592032+0000", 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5109999999999999e-06 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592038+0000", 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7379999999999999e-06 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592042+0000", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628543+0000", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036500828999999999 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628594+0000", 2024-09-17T22:57:48.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1737999999999998e-05 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628651+0000", 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6320999999999998e-05 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5840 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.737 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.039587041000001, 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038232347, 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5840 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592208+0000", 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-06 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592214+0000", 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7479999999999999e-06 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592220+0000", 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.037e-06 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630310+0000", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038090474999999999 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630373+0000", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2467999999999999e-05 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630438+0000", 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5235999999999997e-05 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.741 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5841 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3375104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.039566803, 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038361579999999999, 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.742 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5841 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592228+0000", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592235+0000", 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8040000000000004e-06 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592242+0000", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2230000000000001e-06 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630468+0000", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038225982999999998 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630517+0000", 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8656000000000003e-05 2024-09-17T22:57:48.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630587+0000", 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0698000000000006e-05 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5842 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.039544072999998, 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038469774999999998, 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:48.746 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5842 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594021+0000", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001772849 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594026+0000", 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4399999999999998e-06 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594037+0000", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1071e-05 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594044+0000", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9129999999999997e-06 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630613+0000", 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036569602999999999 2024-09-17T22:57:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630659+0000", 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5246e-05 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630718+0000", 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9653e-05 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:48.750 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:48.751 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:48.751 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:48.752 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:48.752 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:48.752 INFO:teuthology.orchestra.run.smithi159.stdout:1951 writesame 0x7b01a72 thru 0x7b0e346 data_size 0x1(0xc8d5 bytes) 2024-09-17T22:57:48.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:1952 trunc from 0x7b0e347 to 0x151b8c8 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:1953 read 0x13e9ca4 thru 0x13eaabf (0xe1c bytes) 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:1955 read 0x22aba5 thru 0x234478 (0x98d4 bytes) 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.754 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:48.755 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:48.755 INFO:teuthology.orchestra.run.smithi159.stdout:1958 writesame 0x7392bf1 thru 0x739aef0 data_size 0x40(0x8300 bytes) 2024-09-17T22:57:48.755 INFO:teuthology.orchestra.run.smithi159.stdout:1960 read 0x3a1ffcf thru 0x3a2f76d (0xf79f bytes) 2024-09-17T22:57:48.755 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.756 INFO:teuthology.orchestra.run.smithi159.stdout:1963 trunc from 0x739aef1 to 0x287eb42 2024-09-17T22:57:48.756 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:48.756 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:48.756 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:48.757 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1964 write 0xd0adb64 thru 0xd0b2163 (0x4600 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1965 read 0xb83cef3 thru 0xb84884a (0xb958 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1967 read 0x1e42975 thru 0x1e522ea (0xf976 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1969 writesame 0x5ac40b7 thru 0x5acafc6 data_size 0x8(0x6f10 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1972 write 0x80e345b thru 0x80e8d93 (0x5939 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1973 writesame 0x616369b thru 0x6171b9d data_size 0x1(0xe503 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1974 writesame 0x51b4b90 thru 0x51b6d02 data_size 0x1(0x2173 bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1975 read 0x9aa7283 thru 0x9ab452d (0xd2ab bytes) 2024-09-17T22:57:48.758 INFO:teuthology.orchestra.run.smithi159.stdout:1977 trunc from 0xd0b2164 to 0x1431e8 2024-09-17T22:57:48.759 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:48.749+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:48.768 INFO:teuthology.orchestra.run.smithi159.stdout:1978 read 0x52411 thru 0x5496a (0x255a bytes) 2024-09-17T22:57:48.769 INFO:teuthology.orchestra.run.smithi159.stdout:1980 read 0x54696 thru 0x5e648 (0x9fb3 bytes) 2024-09-17T22:57:48.793 INFO:teuthology.orchestra.run.smithi159.stdout:1981 write 0xe82cc9d thru 0xe82cce5 (0x49 bytes) 2024-09-17T22:57:48.796 INFO:teuthology.orchestra.run.smithi159.stdout:1982 write 0x2025846 thru 0x20314f3 (0xbcae bytes) 2024-09-17T22:57:48.798 INFO:teuthology.orchestra.run.smithi159.stdout:1983 writesame 0x98565f3 thru 0x985e102 data_size 0x16(0x7b10 bytes) 2024-09-17T22:57:48.801 INFO:teuthology.orchestra.run.smithi159.stdout:1984 read 0xb4500e8 thru 0xb459f4d (0x9e66 bytes) 2024-09-17T22:57:48.802 INFO:teuthology.orchestra.run.smithi159.stdout:1985 write 0x9249729 thru 0x9249ba4 (0x47c bytes) 2024-09-17T22:57:48.802 INFO:teuthology.orchestra.run.smithi159.stdout:1986 writesame 0x504f318 thru 0x505d129 data_size 0x19(0xde12 bytes) 2024-09-17T22:57:48.806 INFO:teuthology.orchestra.run.smithi159.stdout:1989 read 0x9115416 thru 0x9117c16 (0x2801 bytes) 2024-09-17T22:57:48.806 INFO:teuthology.orchestra.run.smithi159.stdout:1990 write 0x5b3f83e thru 0x5b42846 (0x3009 bytes) 2024-09-17T22:57:48.808 INFO:teuthology.orchestra.run.smithi159.stdout:1991 writesame 0x39af69e thru 0x39b045f data_size 0x3(0xdc2 bytes) 2024-09-17T22:57:48.811 INFO:teuthology.orchestra.run.smithi159.stdout:1992 punch from 0x2ad49c9 to 0x2ae2a8c, (0xe0c3 bytes) 2024-09-17T22:57:48.812 INFO:teuthology.orchestra.run.smithi159.stdout:1993 trunc from 0xe82cce6 to 0xdacafec 2024-09-17T22:57:48.819 INFO:teuthology.orchestra.run.smithi159.stdout:1994 writesame 0x8beffdd thru 0x8bfa0eb data_size 0x1(0xa10f bytes) 2024-09-17T22:57:48.835 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.835 INFO:teuthology.orchestra.run.smithi159.stdout:1996 write 0x7ee78ec thru 0x7eeb34c (0x3a61 bytes) 2024-09-17T22:57:48.838 INFO:teuthology.orchestra.run.smithi159.stdout:1997 punch from 0x69f7ff3 to 0x6a03acc, (0xbad9 bytes) 2024-09-17T22:57:48.838 INFO:teuthology.orchestra.run.smithi159.stdout:1999 read 0x9da17bc thru 0x9da19be (0x203 bytes) 2024-09-17T22:57:48.838 INFO:teuthology.orchestra.run.smithi159.stdout:2000 read 0x2bb4a40 thru 0x2bc12d8 (0xc899 bytes) 2024-09-17T22:57:48.839 INFO:teuthology.orchestra.run.smithi159.stdout:2001 write 0x5d7fbe1 thru 0x5d86085 (0x64a5 bytes) 2024-09-17T22:57:48.840 INFO:teuthology.orchestra.run.smithi159.stdout:2002 punch from 0x1e6c9eb to 0x1e6f07e, (0x2693 bytes) 2024-09-17T22:57:48.840 INFO:teuthology.orchestra.run.smithi159.stdout:2003 read 0x7757f25 thru 0x7761351 (0x942d bytes) 2024-09-17T22:57:48.840 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.840 INFO:teuthology.orchestra.run.smithi159.stdout:2005 write 0x6ec00b9 thru 0x6ec3270 (0x31b8 bytes) 2024-09-17T22:57:48.842 INFO:teuthology.orchestra.run.smithi159.stdout:2006 write 0x4d7b8ef thru 0x4d84890 (0x8fa2 bytes) 2024-09-17T22:57:48.845 INFO:teuthology.orchestra.run.smithi159.stdout:2007 punch from 0x9865387 to 0x9869220, (0x3e99 bytes) 2024-09-17T22:57:48.845 INFO:teuthology.orchestra.run.smithi159.stdout:2008 read 0x634af32 thru 0x634ff4a (0x5019 bytes) 2024-09-17T22:57:48.845 INFO:teuthology.orchestra.run.smithi159.stdout:2010 write 0xa6b481b thru 0xa6befea (0xa7d0 bytes) 2024-09-17T22:57:48.847 INFO:teuthology.orchestra.run.smithi159.stdout:2012 read 0x3338102 thru 0x3341a1d (0x991c bytes) 2024-09-17T22:57:48.848 INFO:teuthology.orchestra.run.smithi159.stdout:2013 write 0x8de2ab8 thru 0x8deead2 (0xc01b bytes) 2024-09-17T22:57:48.850 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.850 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.850 INFO:teuthology.orchestra.run.smithi159.stdout:2016 read 0xaefebb3 thru 0xaf03f82 (0x53d0 bytes) 2024-09-17T22:57:48.852 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:48.850+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:48.858 INFO:teuthology.orchestra.run.smithi159.stdout:2020 trunc from 0xdacafec to 0x6a081d5 2024-09-17T22:57:48.869 INFO:teuthology.orchestra.run.smithi159.stdout:2021 trunc from 0x6a081d5 to 0x628ccad 2024-09-17T22:57:48.870 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:48.870 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:48.875 INFO:teuthology.orchestra.run.smithi159.stdout:2022 write 0x24740f2 thru 0x2481ce5 (0xdbf4 bytes) 2024-09-17T22:57:48.879 INFO:teuthology.orchestra.run.smithi159.stdout:2023 read 0x2d007a3 thru 0x2d0233b (0x1b99 bytes) 2024-09-17T22:57:48.880 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:48.887 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.16 2024-09-17T22:57:48.889 INFO:teuthology.orchestra.run.smithi159.stdout:2026 trunc from 0x628ccad to 0xe4a8c0e 2024-09-17T22:57:48.895 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.895 INFO:teuthology.orchestra.run.smithi159.stdout:2028 write 0x58cd0a6 thru 0x58d2ab5 (0x5a10 bytes) 2024-09-17T22:57:48.896 INFO:teuthology.orchestra.run.smithi159.stdout:2029 write 0x1ed784c thru 0x1ee71ab (0xf960 bytes) 2024-09-17T22:57:48.898 INFO:teuthology.orchestra.run.smithi159.stdout:2030 writesame 0xdbccda2 thru 0xdbda051 data_size 0x8(0xd2b0 bytes) 2024-09-17T22:57:48.903 INFO:teuthology.orchestra.run.smithi159.stdout:2031 punch from 0x4c2ccdf to 0x4c3b645, (0xe966 bytes) 2024-09-17T22:57:48.903 INFO:teuthology.orchestra.run.smithi159.stdout:2032 writesame 0x63dbd95 thru 0x63e9962 data_size 0x55(0xdbce bytes) 2024-09-17T22:57:48.917 INFO:teuthology.orchestra.run.smithi159.stdout:2033 trunc from 0xe4a8c0e to 0x5222234 2024-09-17T22:57:48.928 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:48.938 INFO:teuthology.orchestra.run.smithi159.stdout:2035 trunc from 0x5222234 to 0xd7a42a4 2024-09-17T22:57:48.944 INFO:teuthology.orchestra.run.smithi159.stdout:2037 punch from 0x8e486d4 to 0x8e5168c, (0x8fb8 bytes) 2024-09-17T22:57:48.953 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:48.950+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:48.958 INFO:teuthology.orchestra.run.smithi159.stdout:2038 trunc from 0xd7a42a4 to 0x206245a 2024-09-17T22:57:48.970 INFO:teuthology.orchestra.run.smithi159.stdout:2039 read 0x81d903 thru 0x82c527 (0xec25 bytes) 2024-09-17T22:57:48.970 INFO:teuthology.orchestra.run.smithi159.stdout:2040 trunc from 0x206245a to 0x23994bf 2024-09-17T22:57:48.979 INFO:teuthology.orchestra.run.smithi159.stdout:2041 trunc from 0x23994bf to 0x4613338 2024-09-17T22:57:48.985 INFO:teuthology.orchestra.run.smithi159.stdout:2042 punch from 0x2822b27 to 0x28283b8, (0x5891 bytes) 2024-09-17T22:57:48.985 INFO:teuthology.orchestra.run.smithi159.stdout:2044 read 0x23cdb38 thru 0x23d3702 (0x5bcb bytes) 2024-09-17T22:57:48.987 INFO:teuthology.orchestra.run.smithi159.stdout:2045 trunc from 0x4613338 to 0x6101e38 2024-09-17T22:57:48.991 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:48.991 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:48.991 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:48.991 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:48.991 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:48.993 INFO:teuthology.orchestra.run.smithi159.stdout:2046 write 0x5d6d89a thru 0x5d6de78 (0x5df bytes) 2024-09-17T22:57:49.000 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:49.003 INFO:teuthology.orchestra.run.smithi159.stdout:2047 trunc from 0x6101e38 to 0x69c5e8 2024-09-17T22:57:49.013 INFO:teuthology.orchestra.run.smithi159.stdout:2049 read 0x5e670d thru 0x5f04c8 (0x9dbc bytes) 2024-09-17T22:57:49.035 INFO:teuthology.orchestra.run.smithi159.stdout:2050 writesame 0xd6b65d6 thru 0xd6bfa0f data_size 0x1(0x943a bytes) 2024-09-17T22:57:49.041 INFO:teuthology.orchestra.run.smithi159.stdout:2054 writesame 0xde2d5f6 thru 0xde37372 data_size 0x3(0x9d7d bytes) 2024-09-17T22:57:49.041 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.041 INFO:teuthology.orchestra.run.smithi159.stdout:2056 punch from 0xce562 to 0xd84d0, (0x9f6e bytes) 2024-09-17T22:57:49.041 INFO:teuthology.orchestra.run.smithi159.stdout:2059 writesame 0x2e6d104 thru 0x2e74dd4 data_size 0x1(0x7cd1 bytes) 2024-09-17T22:57:49.052 INFO:teuthology.orchestra.run.smithi159.stdout:2060 write 0x2063f1 thru 0x2155f4 (0xf204 bytes) 2024-09-17T22:57:49.053 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:49.050+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:49.053 INFO:teuthology.orchestra.run.smithi159.stdout:2061 read 0xbd573d thru 0xbe2694 (0xcf58 bytes) 2024-09-17T22:57:49.063 INFO:teuthology.orchestra.run.smithi159.stdout:2062 trunc from 0xde37373 to 0x624c5d0 2024-09-17T22:57:49.070 INFO:teuthology.orchestra.run.smithi159.stdout:2063 read 0x3b7be3e thru 0x3b82a0b (0x6bce bytes) 2024-09-17T22:57:49.070 INFO:teuthology.orchestra.run.smithi159.stdout:2064 write 0x54920b3 thru 0x5498066 (0x5fb4 bytes) 2024-09-17T22:57:49.082 INFO:teuthology.orchestra.run.smithi159.stdout:2065 write 0x8a1fb51 thru 0x8a2c0dd (0xc58d bytes) 2024-09-17T22:57:49.086 INFO:teuthology.orchestra.run.smithi159.stdout:2066 read 0x4e65efa thru 0x4e70f21 (0xb028 bytes) 2024-09-17T22:57:49.090 INFO:teuthology.orchestra.run.smithi159.stdout:2067 trunc from 0x8a2c0de to 0x507c799 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:2068 punch from 0x541e6d to 0x546bcf, (0x4d62 bytes) 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:2069 punch from 0x473b525 to 0x47474ce, (0xbfa9 bytes) 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:2070 punch from 0x15f5f0d to 0x15f636e, (0x461 bytes) 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:2071 read 0x269fda6 thru 0x26ad51c (0xd777 bytes) 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:2073 read 0x1045b08 thru 0x104cad2 (0x6fcb bytes) 2024-09-17T22:57:49.105 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:57:49.107 INFO:teuthology.orchestra.run.smithi159.stdout:2074 write 0x2497417 thru 0x24a4a7e (0xd668 bytes) 2024-09-17T22:57:49.112 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24281 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8767686619999999, 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037645445, 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24281 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232602+0000", 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232605+0000", 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-06 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232612+0000", 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.596e-06 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232623+0000", 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.269238+0000", 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036615385 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270084+0000", 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084551799999999996 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270247+0000", 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000162948 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.117 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24284 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.876740184, 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037880098000000001, 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24284 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:49.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232633+0000", 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4320000000000002e-06 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232639+0000", 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4389999999999997e-06 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232644+0000", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.651e-06 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270285+0000", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037641711000000001 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270368+0000", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2358e-05 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270510+0000", 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014204899999999999 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24285 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1966080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8767200229999998, 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038129107000000002, 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24285 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232652+0000", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1150000000000001e-06 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232659+0000", 2024-09-17T22:57:49.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232663+0000", 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.403e-06 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270554+0000", 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037890946000000002 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270636+0000", 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1919e-05 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270779+0000", 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000143103 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24286 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:49.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8767007929999999, 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038334057999999997, 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24286 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232677+0000", 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1539999999999996e-06 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232683+0000", 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8739999999999997e-06 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232689+0000", 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3299999999999998e-06 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270820+0000", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038131089999999999 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270888+0000", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8288999999999996e-05 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271003+0000", 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114955 2024-09-17T22:57:49.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24289 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.876674049, 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038485336000000002, 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24289 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232698+0000", 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0320000000000002e-06 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232704+0000", 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232713+0000", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2930000000000006e-06 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271036+0000", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038323004000000001 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271078+0000", 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1063999999999999e-05 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271181+0000", 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103559 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24290 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8766498039999999, 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038652381, 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.135 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24290 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232722+0000", 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.561e-06 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232727+0000", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6080000000000003e-06 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232732+0000", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4120000000000003e-06 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271214+0000", 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038481808999999999 2024-09-17T22:57:49.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271268+0000", 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4561999999999997e-05 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271372+0000", 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104101 2024-09-17T22:57:49.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24293 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8766319760000001, 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038796900000000002, 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24293 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:49.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232740+0000", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232744+0000", 2024-09-17T22:57:49.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232747+0000", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.689e-06 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271401+0000", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038653976999999999 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271442+0000", 2024-09-17T22:57:49.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0547999999999999e-05 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271535+0000", 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3034000000000004e-05 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24294 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8766193229999999, 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039269964999999997, 2024-09-17T22:57:49.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24294 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232751+0000", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232752+0000", 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1829999999999999e-06 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232755+0000", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1860000000000001e-06 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232757+0000", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000001e-06 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271598+0000", 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038840787000000002 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271794+0000", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000195434 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272020+0000", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022673999999999999 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24369 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1081344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:49.148 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8762343220000002, 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039126368000000002, 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24369 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233136+0000", 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2099999999999995e-07 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233141+0000", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6380000000000003e-06 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233152+0000", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1405e-05 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233160+0000", 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272065+0000", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905397000000001 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272141+0000", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5465999999999999e-05 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272262+0000", 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000120999 2024-09-17T22:57:49.152 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24370 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8761968439999999, 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039291936999999999, 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.153 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24370 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3900000000000005e-07 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233176+0000", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233188+0000", 2024-09-17T22:57:49.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1535999999999999e-05 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233204+0000", 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5678000000000001e-05 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272290+0000", 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039085904999999997 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272356+0000", 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6587999999999993e-05 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272465+0000", 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108724 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24373 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:49.157 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.876155775, 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039461167999999998, 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24373 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233215+0000", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233217+0000", 2024-09-17T22:57:49.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233224+0000", 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9739999999999996e-06 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233230+0000", 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4809999999999997e-06 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272491+0000", 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039261457 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272560+0000", 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8576000000000002e-05 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272675+0000", 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011552699999999999 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24374 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.876127715, 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039645639000000003, 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24374 2024-09-17T22:57:49.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233243+0000", 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233245+0000", 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6000000000000001e-06 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233252+0000", 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9650000000000002e-06 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233257+0000", 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2549999999999997e-06 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272704+0000", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039446390999999997 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272778+0000", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3818000000000004e-05 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272888+0000", 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110019 2024-09-17T22:57:49.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24377 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1097728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8761044460000003, 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039817298000000001, 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.167 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24377 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233266+0000", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233268+0000", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233275+0000", 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6000000000000003e-06 2024-09-17T22:57:49.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233279+0000", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272918+0000", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039638698 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272980+0000", 2024-09-17T22:57:49.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1463000000000004e-05 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273083+0000", 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010304799999999999 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24378 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8760832459999999, 2024-09-17T22:57:49.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039979973000000002, 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24378 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233289+0000", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233295+0000", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0970000000000002e-06 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233300+0000", 2024-09-17T22:57:49.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7890000000000002e-06 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273107+0000", 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039806766 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273167+0000", 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0541999999999997e-05 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273267+0000", 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9185999999999995e-05 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24381 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8760621240000002, 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040144735000000001, 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.176 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24381 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:49.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233613+0000", 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030540199999999997 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233623+0000", 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.166e-06 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233630+0000", 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8870000000000003e-06 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273292+0000", 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039662289000000003 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273352+0000", 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0600999999999998e-05 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273452+0000", 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9982999999999995e-05 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24382 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1228800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.180 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8757273740000002, 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040027144000000001, 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24382 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:49.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233643+0000", 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233647+0000", 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8210000000000003e-06 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233654+0000", 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3259999999999998e-06 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233660+0000", 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3210000000000004e-06 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273478+0000", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039818781999999997 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273539+0000", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0214999999999999e-05 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273670+0000", 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013101599999999999 2024-09-17T22:57:49.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24385 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8753925159999998, 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039903028, 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24385 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233978+0000", 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233989+0000", 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0685e-05 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234000+0000", 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1282e-05 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234007+0000", 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273697+0000", 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039690053000000003 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273766+0000", 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8483000000000001e-05 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273880+0000", 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011480300000000001 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24386 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8753492669999998, 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040055687, 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24386 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234024+0000", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234035+0000", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1319e-05 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234041+0000", 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8170000000000004e-06 2024-09-17T22:57:49.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273907+0000", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039866255000000003 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273972+0000", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4966000000000001e-05 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274076+0000", 2024-09-17T22:57:49.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104293 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24390 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1122304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8753203940000001, 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040224370000000002, 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.194 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24390 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234050+0000", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234052+0000", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.278e-06 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234059+0000", 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:57:49.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234381+0000", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032158499999999999 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274100+0000", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039719415000000001 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274163+0000", 2024-09-17T22:57:49.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3095000000000005e-05 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274274+0000", 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110373 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24391 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.8749760069999999, 2024-09-17T22:57:49.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040238796, 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24391 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:49.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234395+0000", 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1800000000000005e-07 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234398+0000", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6839999999999998e-06 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234408+0000", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-05 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234415+0000", 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274457+0000", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040042021999999997 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274528+0000", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1940999999999994e-05 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274633+0000", 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104114 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.203 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.203 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:49.205 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:49.205 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:49.201+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:49.206 INFO:teuthology.orchestra.run.smithi159.stdout:2075 write 0xc6d36d4 thru 0xc6d39ac (0x2d9 bytes) 2024-09-17T22:57:49.206 INFO:teuthology.orchestra.run.smithi159.stdout:2076 punch from 0xbc32007 to 0xbc399bb, (0x79b4 bytes) 2024-09-17T22:57:49.206 INFO:teuthology.orchestra.run.smithi159.stdout:2078 writesame 0x200a184 thru 0x2015f50 data_size 0x1(0xbdcd bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2081 read 0x97d89d thru 0x98a57d (0xcce1 bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2082 writesame 0x7e07776 thru 0x7e0e6dc data_size 0x13(0x6f67 bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2084 punch from 0x21dcebc to 0x21e9f46, (0xd08a bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2085 read 0xbdfaa4b thru 0xbdfd97d (0x2f33 bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2087 read 0xd0f1c0 thru 0xd12c16 (0x3a57 bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2090 trunc from 0xc6d39ad to 0x1fcacda 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2091 write 0x7f59611 thru 0x7f5efb3 (0x59a3 bytes) 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.207 INFO:teuthology.orchestra.run.smithi159.stdout:2093 write 0x7e1837a thru 0x7e275a1 (0xf228 bytes) 2024-09-17T22:57:49.208 INFO:teuthology.orchestra.run.smithi159.stdout:2094 write 0xa0687b5 thru 0xa06a2bb (0x1b07 bytes) 2024-09-17T22:57:49.211 INFO:teuthology.orchestra.run.smithi159.stdout:2096 trunc from 0xa06a2bc to 0x9ff11d9 2024-09-17T22:57:49.217 INFO:teuthology.orchestra.run.smithi159.stdout:2097 read 0x141291e thru 0x141c1cb (0x98ae bytes) 2024-09-17T22:57:49.217 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.217 INFO:teuthology.orchestra.run.smithi159.stdout:2099 write 0x876d51f thru 0x877d387 (0xfe69 bytes) 2024-09-17T22:57:49.219 INFO:teuthology.orchestra.run.smithi159.stdout:2100 write 0x5dad318 thru 0x5db33db (0x60c4 bytes) 2024-09-17T22:57:49.221 INFO:teuthology.orchestra.run.smithi159.stdout:2101 write 0x6ac5ef4 thru 0x6ac605f (0x16c bytes) 2024-09-17T22:57:49.224 INFO:teuthology.orchestra.run.smithi159.stdout:2102 write 0x937b93f thru 0x937cf51 (0x1613 bytes) 2024-09-17T22:57:49.227 INFO:teuthology.orchestra.run.smithi159.stdout:2103 write 0x2eb3aff thru 0x2ec0e47 (0xd349 bytes) 2024-09-17T22:57:49.228 INFO:teuthology.orchestra.run.smithi159.stdout:2104 punch from 0x54ebd75 to 0x54f4576, (0x8801 bytes) 2024-09-17T22:57:49.229 INFO:teuthology.orchestra.run.smithi159.stdout:2106 punch from 0x5cf1b02 to 0x5d00812, (0xed10 bytes) 2024-09-17T22:57:49.230 INFO:teuthology.orchestra.run.smithi159.stdout:2107 trunc from 0x9ff11d9 to 0x852c22d 2024-09-17T22:57:49.243 INFO:teuthology.orchestra.run.smithi159.stdout:2108 read 0x781b411 thru 0x781fed9 (0x4ac9 bytes) 2024-09-17T22:57:49.255 INFO:teuthology.orchestra.run.smithi159.stdout:2109 writesame 0xd0d91b0 thru 0xd0e3a3c data_size 0x3(0xa88d bytes) 2024-09-17T22:57:49.255 INFO:teuthology.orchestra.run.smithi159.stdout:2110 punch from 0x8c2d177 to 0x8c362fc, (0x9185 bytes) 2024-09-17T22:57:49.256 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.256 INFO:teuthology.orchestra.run.smithi159.stdout:2112 writesame 0x1f1e343 thru 0x1f23a29 data_size 0x1(0x56e7 bytes) 2024-09-17T22:57:49.265 INFO:teuthology.orchestra.run.smithi159.stdout:2114 writesame 0x1b24468 thru 0x1b2e86f data_size 0x3a(0xa408 bytes) 2024-09-17T22:57:49.269 INFO:teuthology.orchestra.run.smithi159.stdout:2115 trunc from 0xd0e3a3d to 0xa8fe0ff 2024-09-17T22:57:49.283 INFO:teuthology.orchestra.run.smithi159.stdout:2116 write 0xbff7b68 thru 0xbffdafc (0x5f95 bytes) 2024-09-17T22:57:49.286 INFO:teuthology.orchestra.run.smithi159.stdout:2117 read 0x2addd32 thru 0x2aecdf2 (0xf0c1 bytes) 2024-09-17T22:57:49.286 INFO:teuthology.orchestra.run.smithi159.stdout:2119 write 0x3e3b6a7 thru 0x3e49e50 (0xe7aa bytes) 2024-09-17T22:57:49.290 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.290 INFO:teuthology.orchestra.run.smithi159.stdout:2122 read 0xb0da836 thru 0xb0e4b57 (0xa322 bytes) 2024-09-17T22:57:49.290 INFO:teuthology.orchestra.run.smithi159.stdout:2123 punch from 0xa33b716 to 0xa33ee09, (0x36f3 bytes) 2024-09-17T22:57:49.290 INFO:teuthology.orchestra.run.smithi159.stdout:2124 writesame 0x6fbcd85 thru 0x6fc15dc data_size 0x14(0x4858 bytes) 2024-09-17T22:57:49.294 INFO:teuthology.orchestra.run.smithi159.stdout:2126 read 0x7a4515b thru 0x7a51649 (0xc4ef bytes) 2024-09-17T22:57:49.302 INFO:teuthology.orchestra.run.smithi159.stdout:2127 writesame 0xde12bfd thru 0xde1cf4d data_size 0x1(0xa351 bytes) 2024-09-17T22:57:49.302 INFO:teuthology.orchestra.run.smithi159.stdout:2129 writesame 0x8dd04dc thru 0x8dda2c0 data_size 0x1(0x9de5 bytes) 2024-09-17T22:57:49.304 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:49.301+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:49.316 INFO:teuthology.orchestra.run.smithi159.stdout:2130 punch from 0x2f33c7f to 0x2f3abaf, (0x6f30 bytes) 2024-09-17T22:57:49.316 INFO:teuthology.orchestra.run.smithi159.stdout:2132 punch from 0x2848bd1 to 0x284e405, (0x5834 bytes) 2024-09-17T22:57:49.316 INFO:teuthology.orchestra.run.smithi159.stdout:2135 read 0x942d800 thru 0x942d9ae (0x1af bytes) 2024-09-17T22:57:49.316 INFO:teuthology.orchestra.run.smithi159.stdout:2136 writesame 0x95e3800 thru 0x95f108e data_size 0x1(0xd88f bytes) 2024-09-17T22:57:49.345 INFO:teuthology.orchestra.run.smithi159.stdout:2137 write 0x243decd thru 0x2448f53 (0xb087 bytes) 2024-09-17T22:57:49.346 INFO:teuthology.orchestra.run.smithi159.stdout:2138 read 0x206fe47 thru 0x207cb50 (0xcd0a bytes) 2024-09-17T22:57:49.347 INFO:teuthology.orchestra.run.smithi159.stdout:2139 read 0x8f5aac6 thru 0x8f5fc26 (0x5161 bytes) 2024-09-17T22:57:49.347 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.347 INFO:teuthology.orchestra.run.smithi159.stdout:2141 writesame 0xbfed2b2 thru 0xbffcb8d data_size 0x6(0xf8dc bytes) 2024-09-17T22:57:49.353 INFO:teuthology.orchestra.run.smithi159.stdout:2142 read 0x96ebbc4 thru 0x96f489a (0x8cd7 bytes) 2024-09-17T22:57:49.353 INFO:teuthology.orchestra.run.smithi159.stdout:2143 write 0x711885 thru 0x7175c6 (0x5d42 bytes) 2024-09-17T22:57:49.354 INFO:teuthology.orchestra.run.smithi159.stdout:2147 writesame 0x4039db5 thru 0x4045fd8 data_size 0x23(0xc224 bytes) 2024-09-17T22:57:49.355 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:49.355 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:49.355 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:49.358 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:49.358 INFO:teuthology.orchestra.run.smithi159.stdout:2150 clone 18 order 25 su 4194304 sc 3 2024-09-17T22:57:49.364 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:49.406 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:49.402+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:49.506 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:49.503+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:49.507 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:49.508 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:49.517 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:49.607 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:49.604+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406970426000001, 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:57:49.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:57:49.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406954421, 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:49.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406921650999999, 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:57:49.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406893912999999, 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:57:49.677 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:57:49.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406871494000001, 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:57:49.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:57:49.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406849213999999, 2024-09-17T22:57:49.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406641809, 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:57:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:49.695 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.406095194000001, 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:49.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:57:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:57:49.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.405023998000001, 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:49.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.405012543, 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:57:49.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.711 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.405001565999999, 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.712 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:57:49.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:49.716 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.402927434, 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.402915119999999, 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.721 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:49.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.402901883, 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:49.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.402891053999999, 2024-09-17T22:57:49.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:49.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:57:49.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:57:49.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.400932179, 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.735 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:57:49.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.739 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.400907821000001, 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.740 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:57:49.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.744 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.400878603000001, 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:57:49.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:57:49.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:49.749 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.400112034999999, 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:57:49.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:57:49.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.400098933000001, 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:57:49.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:57:49.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:49.758 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:49.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:49.760 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:49.757+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:49.860 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:49.857+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:49.914 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:49.914 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:49.914 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:49.922 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:49.961 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:49.959+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:50.062 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:50.059+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:50.067 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:50.067 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:50.067 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:50.067 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:50.067 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:50.076 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:50.163 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:50.160+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:50.184 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.320451282999997, 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:50.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:57:50.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:57:50.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093767943, 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.191 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:57:50.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:57:50.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:50.195 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093733406999998, 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:57:50.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093679024, 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.200 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:57:50.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:57:50.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093654019999999, 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:57:50.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:57:50.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:57:50.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093633439000001, 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:57:50.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:57:50.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093612155999999, 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:57:50.214 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:57:50.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.218 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093593630000001, 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.219 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:50.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093575458, 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:57:50.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093557629999999, 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:57:50.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:57:50.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.093242496999999, 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.233 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:57:50.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.092520226000001, 2024-09-17T22:57:50.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:57:50.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:57:50.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.092475629999999, 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:57:50.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.246 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.092448308000002, 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:57:50.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.090629509999999, 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.251 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:57:50.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.090286705, 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:57:50.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.260 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.088015573, 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:57:50.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.087983478999998, 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:57:50.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.087956941000002, 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:57:50.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:57:50.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:50.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.087938048000002, 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:57:50.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:57:50.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:50.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:50.279 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:50.280 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:50.277+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:50.281 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:50.303 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T22:57:50.328 INFO:teuthology.orchestra.run.smithi159.stdout:2151 read 0xcb34ecb thru 0xcb36ccb (0x1e01 bytes) 2024-09-17T22:57:50.328 INFO:teuthology.orchestra.run.smithi159.stdout:2152 trunc from 0xde1cf4e to 0xd9996b5 2024-09-17T22:57:50.339 INFO:teuthology.orchestra.run.smithi159.stdout:2153 write 0x621af2a thru 0x6220b7d (0x5c54 bytes) 2024-09-17T22:57:50.343 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:50.343 INFO:teuthology.orchestra.run.smithi159.stdout:2157 writesame 0x2578d34 thru 0x2586ad3 data_size 0x30(0xdda0 bytes) 2024-09-17T22:57:50.349 INFO:teuthology.orchestra.run.smithi159.stdout:2158 writesame 0x1aab06 thru 0x1b3433 data_size 0x9(0x892e bytes) 2024-09-17T22:57:50.352 INFO:teuthology.orchestra.run.smithi159.stdout:2159 punch from 0x49b4ada to 0x49c10a5, (0xc5cb bytes) 2024-09-17T22:57:50.364 INFO:teuthology.orchestra.run.smithi159.stdout:2160 trunc from 0xd9996b5 to 0x36678b0 2024-09-17T22:57:50.378 INFO:teuthology.orchestra.run.smithi159.stdout:2162 punch from 0x979abf to 0x97b26c, (0x17ad bytes) 2024-09-17T22:57:50.378 INFO:teuthology.orchestra.run.smithi159.stdout:2169 punch from 0x1ce614c to 0x1ce8054, (0x1f08 bytes) 2024-09-17T22:57:50.382 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:50.379+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:50.388 INFO:teuthology.orchestra.run.smithi159.stdout:2170 writesame 0x7d3ecee thru 0x7d4ac9b data_size 0x23(0xbfae bytes) 2024-09-17T22:57:50.389 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:50.389 INFO:teuthology.orchestra.run.smithi159.stdout:2172 read 0x729831e thru 0x7299609 (0x12ec bytes) 2024-09-17T22:57:50.389 INFO:teuthology.orchestra.run.smithi159.stdout:2173 read 0x3d7c224 thru 0x3d83aa7 (0x7884 bytes) 2024-09-17T22:57:50.394 INFO:teuthology.orchestra.run.smithi159.stdout:2176 write 0x81d3750 thru 0x81e0d03 (0xd5b4 bytes) 2024-09-17T22:57:50.397 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:50.397 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:50.401 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:50.402 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:50.402 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:50.402 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:50.409 INFO:teuthology.orchestra.run.smithi159.stdout:2180 writesame 0xdc3c5d1 thru 0xdc41118 data_size 0x49(0x4b48 bytes) 2024-09-17T22:57:50.413 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:50.414 INFO:teuthology.orchestra.run.smithi159.stdout:2182 trunc from 0xdc41119 to 0x9aba5d6 2024-09-17T22:57:50.430 INFO:teuthology.orchestra.run.smithi159.stdout:2186 trunc from 0x9aba5d6 to 0xf2c13a 2024-09-17T22:57:50.441 INFO:teuthology.orchestra.run.smithi159.stdout:2187 punch from 0x8105c7 to 0x81baad, (0xb4e6 bytes) 2024-09-17T22:57:50.441 INFO:teuthology.orchestra.run.smithi159.stdout:2189 clone 19 order 23 su 262144 sc 14 2024-09-17T22:57:50.483 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:50.480+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:50.521 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:50.522 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:50.522 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:50.522 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:50.522 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:50.532 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:50.616 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:50.614+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:50.677 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409435926, 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.678 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:57:50.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.682 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409331890000001, 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:57:50.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:57:50.686 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409310629, 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:50.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409289286, 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.691 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:50.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:57:50.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409219829, 2024-09-17T22:57:50.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:50.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:57:50.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409200851, 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:57:50.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:57:50.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409171922000001, 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.704 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409145984, 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.708 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:57:50.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.712 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.40911799, 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:57:50.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409097065999999, 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.717 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409050325000001, 2024-09-17T22:57:50.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:57:50.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409027917, 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.726 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:57:50.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.729 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.409004573000001, 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.730 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:57:50.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408981883999999, 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.734 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:57:50.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.740 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.740 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:50.740 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408959684999999, 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:57:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408930899, 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:57:50.748 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408908265999999, 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.749 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408889013, 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:57:50.753 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:57:50.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:50.757 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.408861837, 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 6.2312228420000002, 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:50.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:57:50.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:57:50.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:57:50.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:50.768 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:50.769 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:50.770 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:50.771 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:50.767+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:50.832 INFO:teuthology.orchestra.run.smithi171.stdout: marking collection for removal 2024-09-17T22:57:50.832 INFO:teuthology.orchestra.run.smithi171.stdout:setting '_remove' omap key 2024-09-17T22:57:50.832 INFO:teuthology.orchestra.run.smithi171.stdout:finish_remove_pgs 3.16_head removing 3.16 2024-09-17T22:57:50.833 INFO:teuthology.orchestra.run.smithi171.stdout:Remove successful 2024-09-17T22:57:50.871 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:50.869+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:50.917 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:50.918 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:50.918 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:50.918 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:50.926 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:50.932 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:50.933 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:50.950 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:50.950 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:50.966 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:50.972 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:50.969+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:50.983 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:50.983 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:50.999 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:51.000 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:51.000 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:51.016 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:51.016 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:51.033 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:51.034 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:51.037 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:51.037 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:51.038 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.038 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:51.090 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:51.087+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:51.171 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone18 from 0xf2c13a (overlap 0xf2c13a) to 0x184685 2024-09-17T22:57:51.175 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:51.172+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:51.205 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.205 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.205 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:57:51.215 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:51.219 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T22:57:51.276 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:51.273+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:51.340 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.341 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.16.5 2024-09-17T22:57:51.354 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:51.400 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:51.397+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:51.464 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.464 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.472 INFO:teuthology.orchestra.run.smithi159.stdout:2192 punch from 0x6a97eb to 0x6b57df, (0xbff4 bytes) 2024-09-17T22:57:51.473 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:51.475 INFO:teuthology.orchestra.run.smithi159.stdout:2194 punch from 0xcdb244 to 0xce1c54, (0x6a10 bytes) 2024-09-17T22:57:51.475 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:51.478 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:51.475+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:51.488 INFO:teuthology.orchestra.run.smithi159.stdout:2196 trunc from 0xf2c13a to 0xc745d5a 2024-09-17T22:57:51.494 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:51.506 INFO:teuthology.orchestra.run.smithi159.stdout:2198 trunc from 0xc745d5a to 0x25d4ca9 2024-09-17T22:57:51.553 INFO:teuthology.orchestra.run.smithi159.stdout:2199 write 0xab8c02d thru 0xab90c88 (0x4c5c bytes) 2024-09-17T22:57:51.557 INFO:teuthology.orchestra.run.smithi159.stdout:2200 read 0x90fb731 thru 0x9102b05 (0x73d5 bytes) 2024-09-17T22:57:51.566 INFO:teuthology.orchestra.run.smithi159.stdout:2201 trunc from 0xab90c89 to 0x3aa5a0c 2024-09-17T22:57:51.578 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:51.576+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:51.582 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:51.582 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:51.582 INFO:teuthology.orchestra.run.smithi159.stdout:2204 punch from 0x1d2e597 to 0x1d3005c, (0x1ac5 bytes) 2024-09-17T22:57:51.583 INFO:teuthology.orchestra.run.smithi159.stdout:2205 writesame 0x241f92c thru 0x2422f32 data_size 0x1(0x3607 bytes) 2024-09-17T22:57:51.589 INFO:teuthology.orchestra.run.smithi159.stdout:2206 read 0x251a449 thru 0x25284d5 (0xe08d bytes) 2024-09-17T22:57:51.597 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.597 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.600 INFO:teuthology.orchestra.run.smithi159.stdout:2207 trunc from 0x3aa5a0c to 0xccef6b7 2024-09-17T22:57:51.605 INFO:teuthology.orchestra.run.smithi159.stdout:2208 clone 20 order 24 su 131072 sc 15 2024-09-17T22:57:51.608 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:51.692 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:51.690+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:51.752 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.752 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.761 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:51.801 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:51.799+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:51.882 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:51.880+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:51.904 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:51.905 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:51.913 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:51.984 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:51.981+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:52.058 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.058 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:52.110 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:52.108+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:52.114 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:57:52.182 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone19 from 0xccef6b7 (overlap 0xf2c13a) to 0x79609d 2024-09-17T22:57:52.183 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.183 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.185 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:52.182+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:52.194 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:52.259 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T22:57:52.264 INFO:teuthology.orchestra.run.smithi159.stdout:2209 write 0x721aadd thru 0x7222835 (0x7d59 bytes) 2024-09-17T22:57:52.275 INFO:teuthology.orchestra.run.smithi159.stdout:2210 read 0x8e1e9c0 thru 0x8e1fa70 (0x10b1 bytes) 2024-09-17T22:57:52.275 INFO:teuthology.orchestra.run.smithi159.stdout:2212 write 0x76fd885 thru 0x76fe24f (0x9cb bytes) 2024-09-17T22:57:52.280 INFO:teuthology.orchestra.run.smithi159.stdout:2213 punch from 0x2c42c1b to 0x2c4de03, (0xb1e8 bytes) 2024-09-17T22:57:52.280 INFO:teuthology.orchestra.run.smithi159.stdout:2214 read 0xad4fe23 thru 0xad51897 (0x1a75 bytes) 2024-09-17T22:57:52.281 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.281 INFO:teuthology.orchestra.run.smithi159.stdout:2217 read 0x4e96e35 thru 0x4ea6008 (0xf1d4 bytes) 2024-09-17T22:57:52.286 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:52.283+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:52.287 INFO:teuthology.orchestra.run.smithi159.stdout:2218 writesame 0xd7acce0 thru 0xd7b8030 data_size 0x1(0xb351 bytes) 2024-09-17T22:57:52.298 INFO:teuthology.orchestra.run.smithi159.stdout:2219 write 0xe84f29b thru 0xe85e63c (0xf3a2 bytes) 2024-09-17T22:57:52.299 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.314 INFO:teuthology.orchestra.run.smithi159.stdout:2221 trunc from 0xe85e63d to 0x2767a6b 2024-09-17T22:57:52.356 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.356 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.362 INFO:teuthology.orchestra.run.smithi159.stdout:2223 writesame 0x54a20b4 thru 0x54a8c05 data_size 0x39(0x6b52 bytes) 2024-09-17T22:57:52.365 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:52.368 INFO:teuthology.orchestra.run.smithi159.stdout:2225 trunc from 0x54a8c06 to 0x95b71f2 2024-09-17T22:57:52.374 INFO:teuthology.orchestra.run.smithi159.stdout:2226 punch from 0x1f7b9c to 0x20414f, (0xc5b3 bytes) 2024-09-17T22:57:52.384 INFO:teuthology.orchestra.run.smithi159.stdout:2227 trunc from 0x95b71f2 to 0xd5de90 2024-09-17T22:57:52.395 INFO:teuthology.orchestra.run.smithi159.stdout:2229 read 0x3e7e01 thru 0x3f12a6 (0x94a6 bytes) 2024-09-17T22:57:52.395 INFO:teuthology.orchestra.run.smithi159.stdout:2230 read 0x191b26 thru 0x1951e9 (0x36c4 bytes) 2024-09-17T22:57:52.395 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.395 INFO:teuthology.orchestra.run.smithi159.stdout:2234 read 0x190c6a thru 0x196942 (0x5cd9 bytes) 2024-09-17T22:57:52.406 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:52.403+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:52.414 INFO:teuthology.orchestra.run.smithi159.stdout:2236 writesame 0xc33a64d thru 0xc33b170 data_size 0x4(0xb24 bytes) 2024-09-17T22:57:52.422 INFO:teuthology.orchestra.run.smithi159.stdout:2237 write 0xd7d787a thru 0xd7df23c (0x79c3 bytes) 2024-09-17T22:57:52.423 INFO:teuthology.orchestra.run.smithi159.stdout:2238 read 0x6413f6d thru 0x641e660 (0xa6f4 bytes) 2024-09-17T22:57:52.424 INFO:teuthology.orchestra.run.smithi159.stdout:2239 trunc from 0xd7df23d to 0xc58cb10 2024-09-17T22:57:52.432 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.432 INFO:teuthology.orchestra.run.smithi159.stdout:2243 write 0x6ea2a30 thru 0x6eae0e5 (0xb6b6 bytes) 2024-09-17T22:57:52.434 INFO:teuthology.orchestra.run.smithi159.stdout:2244 read 0x87d8eea thru 0x87da7c2 (0x18d9 bytes) 2024-09-17T22:57:52.434 INFO:teuthology.orchestra.run.smithi159.stdout:2246 punch from 0x84db770 to 0x84dfd99, (0x4629 bytes) 2024-09-17T22:57:52.435 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.435 INFO:teuthology.orchestra.run.smithi159.stdout:2250 read 0x879333d thru 0x879f2df (0xbfa3 bytes) 2024-09-17T22:57:52.435 INFO:teuthology.orchestra.run.smithi159.stdout:2252 read 0x41a9183 thru 0x41b6382 (0xd200 bytes) 2024-09-17T22:57:52.435 INFO:teuthology.orchestra.run.smithi159.stdout:2253 punch from 0x665b1f0 to 0x665df3f, (0x2d4f bytes) 2024-09-17T22:57:52.435 INFO:teuthology.orchestra.run.smithi159.stdout:2255 read 0x2ccaafa thru 0x2ccca4f (0x1f56 bytes) 2024-09-17T22:57:52.437 INFO:teuthology.orchestra.run.smithi159.stdout:2256 trunc from 0xc58cb10 to 0xe8c1b27 2024-09-17T22:57:52.443 INFO:teuthology.orchestra.run.smithi159.stdout:2257 write 0xbf90a0c thru 0xbf94cde (0x42d3 bytes) 2024-09-17T22:57:52.445 INFO:teuthology.orchestra.run.smithi159.stdout:2260 writesame 0x3636d0f thru 0x3646b0b data_size 0x17(0xfdfd bytes) 2024-09-17T22:57:52.447 INFO:teuthology.orchestra.run.smithi159.stdout:2262 punch from 0xa29bb1e to 0xa2a6b3b, (0xb01d bytes) 2024-09-17T22:57:52.447 INFO:teuthology.orchestra.run.smithi159.stdout:2265 writesame 0x1efe618 thru 0x1f0bb75 data_size 0x1f(0xd55e bytes) 2024-09-17T22:57:52.449 INFO:teuthology.orchestra.run.smithi159.stdout:2266 write 0x3530732 thru 0x3534ea7 (0x4776 bytes) 2024-09-17T22:57:52.450 INFO:teuthology.orchestra.run.smithi159.stdout:2267 write 0x2a7c766 thru 0x2a83fcb (0x7866 bytes) 2024-09-17T22:57:52.452 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.452 INFO:teuthology.orchestra.run.smithi159.stdout:2269 write 0xe0ab842 thru 0xe0ae576 (0x2d35 bytes) 2024-09-17T22:57:52.453 INFO:teuthology.orchestra.run.smithi159.stdout:2270 write 0x3f22a44 thru 0x3f245be (0x1b7b bytes) 2024-09-17T22:57:52.454 INFO:teuthology.orchestra.run.smithi159.stdout:2273 read 0xe62ac19 thru 0xe62dc91 (0x3079 bytes) 2024-09-17T22:57:52.472 INFO:teuthology.orchestra.run.smithi159.stdout:2274 trunc from 0xe8c1b27 to 0x2679db 2024-09-17T22:57:52.488 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:52.485+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:52.493 INFO:teuthology.orchestra.run.smithi159.stdout:2275 writesame 0x73b7c91 thru 0x73bd7e6 data_size 0x1b(0x5b56 bytes) 2024-09-17T22:57:52.504 INFO:teuthology.orchestra.run.smithi159.stdout:2276 write 0xacd0ce4 thru 0xacd866b (0x7988 bytes) 2024-09-17T22:57:52.505 INFO:teuthology.orchestra.run.smithi159.stdout:2278 write 0xaacb932 thru 0xaad6160 (0xa82f bytes) 2024-09-17T22:57:52.507 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.507 INFO:teuthology.orchestra.run.smithi159.stdout:2281 writesame 0x49b81f2 thru 0x49c0229 data_size 0x2c(0x8038 bytes) 2024-09-17T22:57:52.508 INFO:teuthology.orchestra.run.smithi159.stdout:2282 write 0x459ffe5 thru 0x45adc1f (0xdc3b bytes) 2024-09-17T22:57:52.510 INFO:teuthology.orchestra.run.smithi159.stdout:2283 punch from 0x21e7f5c to 0x21f2cc3, (0xad67 bytes) 2024-09-17T22:57:52.510 INFO:teuthology.orchestra.run.smithi159.stdout:2285 read 0x3fd191e thru 0x3fd7743 (0x5e26 bytes) 2024-09-17T22:57:52.512 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.513 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.521 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:52.522 INFO:teuthology.orchestra.run.smithi159.stdout:2286 trunc from 0xacd866c to 0x78ede5 2024-09-17T22:57:52.529 INFO:teuthology.orchestra.run.smithi159.stdout:2287 read 0x7b922 thru 0x80785 (0x4e64 bytes) 2024-09-17T22:57:52.530 INFO:teuthology.orchestra.run.smithi159.stdout:2288 read 0x17015d thru 0x17d2d6 (0xd17a bytes) 2024-09-17T22:57:52.530 INFO:teuthology.orchestra.run.smithi159.stdout:2290 read 0x70a0fa thru 0x718848 (0xe74f bytes) 2024-09-17T22:57:52.552 INFO:teuthology.orchestra.run.smithi159.stdout:2291 writesame 0xe972d4e thru 0xe979175 data_size 0x14(0x6428 bytes) 2024-09-17T22:57:52.553 INFO:teuthology.orchestra.run.smithi159.stdout:2292 punch from 0x3610601 to 0x361ddc7, (0xd7c6 bytes) 2024-09-17T22:57:52.553 INFO:teuthology.orchestra.run.smithi159.stdout:2294 write 0x7e068fc thru 0x7e0a468 (0x3b6d bytes) 2024-09-17T22:57:52.554 INFO:teuthology.orchestra.run.smithi159.stdout:2295 write 0xbf33284 thru 0xbf37588 (0x4305 bytes) 2024-09-17T22:57:52.556 INFO:teuthology.orchestra.run.smithi159.stdout:2297 punch from 0x23b0e2a to 0x23be01c, (0xd1f2 bytes) 2024-09-17T22:57:52.556 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.556 INFO:teuthology.orchestra.run.smithi159.stdout:2299 read 0x312b6cb thru 0x312e111 (0x2a47 bytes) 2024-09-17T22:57:52.556 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.557 INFO:teuthology.orchestra.run.smithi159.stdout:2303 read 0x40caa60 thru 0x40d8b1f (0xe0c0 bytes) 2024-09-17T22:57:52.557 INFO:teuthology.orchestra.run.smithi159.stdout:2304 read 0x7467735 thru 0x746a164 (0x2a30 bytes) 2024-09-17T22:57:52.557 INFO:teuthology.orchestra.run.smithi159.stdout:2305 read 0x1aa3c6c thru 0x1aae63c (0xa9d1 bytes) 2024-09-17T22:57:52.557 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.557 INFO:teuthology.orchestra.run.smithi159.stdout:2308 write 0xb8ae495 thru 0xb8b18aa (0x3416 bytes) 2024-09-17T22:57:52.558 INFO:teuthology.orchestra.run.smithi159.stdout:2309 write 0x5328b31 thru 0x5337841 (0xed11 bytes) 2024-09-17T22:57:52.572 INFO:teuthology.orchestra.run.smithi159.stdout:2310 trunc from 0xe979176 to 0x486959c 2024-09-17T22:57:52.590 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:52.586+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:52.593 INFO:teuthology.orchestra.run.smithi159.stdout:2311 write 0xc2e3e16 thru 0xc2ed1d4 (0x93bf bytes) 2024-09-17T22:57:52.594 INFO:teuthology.orchestra.run.smithi159.stdout:2312 read 0xb4a2865 thru 0xb4a5407 (0x2ba3 bytes) 2024-09-17T22:57:52.594 INFO:teuthology.orchestra.run.smithi159.stdout:2313 read 0x4c1bee0 thru 0x4c23be7 (0x7d08 bytes) 2024-09-17T22:57:52.595 INFO:teuthology.orchestra.run.smithi159.stdout:2315 read 0x1e21d77 thru 0x1e28e5c (0x70e6 bytes) 2024-09-17T22:57:52.596 INFO:teuthology.orchestra.run.smithi159.stdout:2316 read 0x88fc0bb thru 0x8901eff (0x5e45 bytes) 2024-09-17T22:57:52.596 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.596 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.610 INFO:teuthology.orchestra.run.smithi159.stdout:2320 trunc from 0xc2ed1d5 to 0x306e48 2024-09-17T22:57:52.631 INFO:teuthology.orchestra.run.smithi159.stdout:2321 write 0x603b652 thru 0x604a8fe (0xf2ad bytes) 2024-09-17T22:57:52.631 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.631 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.641 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:52.641 INFO:teuthology.orchestra.run.smithi159.stdout:2322 writesame 0x8630d95 thru 0x863756d data_size 0xd(0x67d9 bytes) 2024-09-17T22:57:52.643 INFO:teuthology.orchestra.run.smithi159.stdout:2323 read 0x7761c80 thru 0x77661a3 (0x4524 bytes) 2024-09-17T22:57:52.643 INFO:teuthology.orchestra.run.smithi159.stdout:2324 write 0x8cf981 thru 0x8daf5c (0xb5dc bytes) 2024-09-17T22:57:52.645 INFO:teuthology.orchestra.run.smithi159.stdout:2325 punch from 0x2cbda4 to 0x2d585f, (0x9abb bytes) 2024-09-17T22:57:52.645 INFO:teuthology.orchestra.run.smithi159.stdout:2327 read 0x6084faa thru 0x608f4d2 (0xa529 bytes) 2024-09-17T22:57:52.645 INFO:teuthology.orchestra.run.smithi159.stdout:2328 punch from 0x717b2e6 to 0x717c4b5, (0x11cf bytes) 2024-09-17T22:57:52.645 INFO:teuthology.orchestra.run.smithi159.stdout:2329 writesame 0x12a5a0b thru 0x12b273e data_size 0x2e(0xcd34 bytes) 2024-09-17T22:57:52.647 INFO:teuthology.orchestra.run.smithi159.stdout:2330 write 0x7959d28 thru 0x795e9a2 (0x4c7b bytes) 2024-09-17T22:57:52.648 INFO:teuthology.orchestra.run.smithi159.stdout:2331 read 0x2aeb170 thru 0x2aedb0e (0x299f bytes) 2024-09-17T22:57:52.648 INFO:teuthology.orchestra.run.smithi159.stdout:2332 read 0x624f74 thru 0x63460d (0xf69a bytes) 2024-09-17T22:57:52.649 INFO:teuthology.orchestra.run.smithi159.stdout:2333 write 0x6994b03 thru 0x6994b32 (0x30 bytes) 2024-09-17T22:57:52.650 INFO:teuthology.orchestra.run.smithi159.stdout:2334 read 0x4d35e15 thru 0x4d3df0d (0x80f9 bytes) 2024-09-17T22:57:52.650 INFO:teuthology.orchestra.run.smithi159.stdout:2336 write 0x200d712 thru 0x201b190 (0xda7f bytes) 2024-09-17T22:57:52.651 INFO:teuthology.orchestra.run.smithi159.stdout:2337 punch from 0x45bcc06 to 0x45c1a09, (0x4e03 bytes) 2024-09-17T22:57:52.658 INFO:teuthology.orchestra.run.smithi159.stdout:2338 write 0x9b9bcc9 thru 0x9ba9cee (0xe026 bytes) 2024-09-17T22:57:52.668 INFO:teuthology.orchestra.run.smithi159.stdout:2339 trunc from 0x9ba9cef to 0x2daddc6 2024-09-17T22:57:52.675 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.690 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:52.688+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:52.694 INFO:teuthology.orchestra.run.smithi159.stdout:2342 writesame 0xe6129a7 thru 0xe617dbc data_size 0x2f(0x5416 bytes) 2024-09-17T22:57:52.695 INFO:teuthology.orchestra.run.smithi159.stdout:2345 read 0xe2c2e8f thru 0xe2c71af (0x4321 bytes) 2024-09-17T22:57:52.700 INFO:teuthology.orchestra.run.smithi159.stdout:2346 write 0xed30a48 thru 0xed37cce (0x7287 bytes) 2024-09-17T22:57:52.701 INFO:teuthology.orchestra.run.smithi159.stdout:2347 writesame 0x9828e56 thru 0x982bdb0 data_size 0x9(0x2f5b bytes) 2024-09-17T22:57:52.703 INFO:teuthology.orchestra.run.smithi159.stdout:2348 read 0x88db7e1 thru 0x88e747b (0xbc9b bytes) 2024-09-17T22:57:52.703 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.704 INFO:teuthology.orchestra.run.smithi159.stdout:2352 write 0x87e0843 thru 0x87f01c0 (0xf97e bytes) 2024-09-17T22:57:52.705 INFO:teuthology.orchestra.run.smithi159.stdout:2353 read 0x2e93d36 thru 0x2e9fd2e (0xbff9 bytes) 2024-09-17T22:57:52.705 INFO:teuthology.orchestra.run.smithi159.stdout:2354 write 0x9c32f01 thru 0x9c423c9 (0xf4c9 bytes) 2024-09-17T22:57:52.706 INFO:teuthology.orchestra.run.smithi159.stdout:2356 read 0xecb13b thru 0xed04f5 (0x53bb bytes) 2024-09-17T22:57:52.707 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.707 INFO:teuthology.orchestra.run.smithi159.stdout:2361 punch from 0xca12808 to 0xca1dfa5, (0xb79d bytes) 2024-09-17T22:57:52.707 INFO:teuthology.orchestra.run.smithi159.stdout:2362 read 0x1ccbc53 thru 0x1cd5c74 (0xa022 bytes) 2024-09-17T22:57:52.713 INFO:teuthology.orchestra.run.smithi159.stdout:2363 trunc from 0xed37ccf to 0xa4f498b 2024-09-17T22:57:52.721 INFO:teuthology.orchestra.run.smithi159.stdout:2364 read 0x62a123f thru 0x62acd75 (0xbb37 bytes) 2024-09-17T22:57:52.722 INFO:teuthology.orchestra.run.smithi159.stdout:2365 read 0x66b166 thru 0x66ca48 (0x18e3 bytes) 2024-09-17T22:57:52.722 INFO:teuthology.orchestra.run.smithi159.stdout:2366 read 0x8b7b7db thru 0x8b7def6 (0x271c bytes) 2024-09-17T22:57:52.730 INFO:teuthology.orchestra.run.smithi159.stdout:2367 trunc from 0xa4f498b to 0x3e192b0 2024-09-17T22:57:52.750 INFO:teuthology.orchestra.run.smithi159.stdout:2368 write 0xa0a969c thru 0xa0afdbd (0x6722 bytes) 2024-09-17T22:57:52.752 INFO:teuthology.orchestra.run.smithi159.stdout:2369 writesame 0x379f4e6 thru 0x37ac704 data_size 0x1(0xd21f bytes) 2024-09-17T22:57:52.787 INFO:teuthology.orchestra.run.smithi159.stdout:2370 write 0xe701e03 thru 0xe703589 (0x1787 bytes) 2024-09-17T22:57:52.793 INFO:teuthology.orchestra.run.smithi159.stdout:2373 trunc from 0xe70358a to 0xbfe5df6 2024-09-17T22:57:52.793 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:52.790+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:52.793 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.794 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.803 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:52.807 INFO:teuthology.orchestra.run.smithi159.stdout:2374 writesame 0xd49bafc thru 0xd49cf1a data_size 0x11(0x141f bytes) 2024-09-17T22:57:52.810 INFO:teuthology.orchestra.run.smithi159.stdout:2376 write 0xd2e59a1 thru 0xd2f544e (0xfaae bytes) 2024-09-17T22:57:52.810 INFO:teuthology.orchestra.run.smithi159.stdout:2378 punch from 0x8dce28d to 0x8dd06f3, (0x2466 bytes) 2024-09-17T22:57:52.822 INFO:teuthology.orchestra.run.smithi159.stdout:2379 trunc from 0xd49cf1b to 0x28c364f 2024-09-17T22:57:52.830 INFO:teuthology.orchestra.run.smithi159.stdout:2380 read 0x148a899 thru 0x1492748 (0x7eb0 bytes) 2024-09-17T22:57:52.831 INFO:teuthology.orchestra.run.smithi159.stdout:2382 trunc from 0x28c364f to 0x1a4d38b 2024-09-17T22:57:52.840 INFO:teuthology.orchestra.run.smithi159.stdout:2383 punch from 0x1231622 to 0x123b526, (0x9f04 bytes) 2024-09-17T22:57:52.841 INFO:teuthology.orchestra.run.smithi159.stdout:2384 punch from 0x628107 to 0x6354e3, (0xd3dc bytes) 2024-09-17T22:57:52.841 INFO:teuthology.orchestra.run.smithi159.stdout:2386 punch from 0x599655 to 0x59afed, (0x1998 bytes) 2024-09-17T22:57:52.841 INFO:teuthology.orchestra.run.smithi159.stdout:2387 read 0xfcccbb thru 0xfd4b27 (0x7e6d bytes) 2024-09-17T22:57:52.853 INFO:teuthology.orchestra.run.smithi159.stdout:2388 write 0x7cd4ce5 thru 0x7cd63c3 (0x16df bytes) 2024-09-17T22:57:52.855 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.860 INFO:teuthology.orchestra.run.smithi159.stdout:2390 trunc from 0x7cd63c4 to 0x3a83f3b 2024-09-17T22:57:52.874 INFO:teuthology.orchestra.run.smithi159.stdout:2391 write 0x41d2c3e thru 0x41d5349 (0x270c bytes) 2024-09-17T22:57:52.882 INFO:teuthology.orchestra.run.smithi159.stdout:2393 write 0x503f176 thru 0x50469b3 (0x783e bytes) 2024-09-17T22:57:52.883 INFO:teuthology.orchestra.run.smithi159.stdout:2394 punch from 0x33d1690 to 0x33d2861, (0x11d1 bytes) 2024-09-17T22:57:52.883 INFO:teuthology.orchestra.run.smithi159.stdout:2395 read 0x191f756 thru 0x19248c2 (0x516d bytes) 2024-09-17T22:57:52.884 INFO:teuthology.orchestra.run.smithi159.stdout:2396 read 0xb53ccc thru 0xb63937 (0xfc6c bytes) 2024-09-17T22:57:52.884 INFO:teuthology.orchestra.run.smithi159.stdout:2397 read 0x15b7b95 thru 0x15b852a (0x996 bytes) 2024-09-17T22:57:52.884 INFO:teuthology.orchestra.run.smithi159.stdout:2398 write 0x3173036 thru 0x317d050 (0xa01b bytes) 2024-09-17T22:57:52.886 INFO:teuthology.orchestra.run.smithi159.stdout:2399 punch from 0x2fe5989 to 0x2ff2568, (0xcbdf bytes) 2024-09-17T22:57:52.886 INFO:teuthology.orchestra.run.smithi159.stdout:2400 punch from 0x98407c to 0x99054d, (0xc4d1 bytes) 2024-09-17T22:57:52.886 INFO:teuthology.orchestra.run.smithi159.stdout:2402 punch from 0x8fd461 to 0x90c0a3, (0xec42 bytes) 2024-09-17T22:57:52.886 INFO:teuthology.orchestra.run.smithi159.stdout:2403 read 0x2e4375b thru 0x2e51926 (0xe1cc bytes) 2024-09-17T22:57:52.893 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:52.891+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:52.899 INFO:teuthology.orchestra.run.smithi159.stdout:2404 write 0xb078bb8 thru 0xb07ca1f (0x3e68 bytes) 2024-09-17T22:57:52.900 INFO:teuthology.orchestra.run.smithi159.stdout:2405 write 0xa70bb49 thru 0xa710026 (0x44de bytes) 2024-09-17T22:57:52.901 INFO:teuthology.orchestra.run.smithi159.stdout:2407 write 0xd4324b thru 0xd4cbb5 (0x996b bytes) 2024-09-17T22:57:52.902 INFO:teuthology.orchestra.run.smithi159.stdout:2408 read 0x3ff48cb thru 0x3ff8728 (0x3e5e bytes) 2024-09-17T22:57:52.902 INFO:teuthology.orchestra.run.smithi159.stdout:2409 write 0x6c7bfa thru 0x6d0a7f (0x8e86 bytes) 2024-09-17T22:57:52.904 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.904 INFO:teuthology.orchestra.run.smithi159.stdout:2412 writesame 0x58f5d5b thru 0x58ffc0a data_size 0x8(0x9eb0 bytes) 2024-09-17T22:57:52.907 INFO:teuthology.orchestra.run.smithi159.stdout:2414 read 0x7b095e8 thru 0x7b14949 (0xb362 bytes) 2024-09-17T22:57:52.908 INFO:teuthology.orchestra.run.smithi159.stdout:2415 writesame 0x8c5d83c thru 0x8c5ef74 data_size 0x1d(0x1739 bytes) 2024-09-17T22:57:52.920 INFO:teuthology.orchestra.run.smithi159.stdout:2416 write 0xeaf9362 thru 0xeb081e4 (0xee83 bytes) 2024-09-17T22:57:52.921 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.923 INFO:teuthology.orchestra.run.smithi159.stdout:2420 trunc from 0xeb081e5 to 0xd8fbe68 2024-09-17T22:57:52.931 INFO:teuthology.orchestra.run.smithi159.stdout:2422 write 0x90c8405 thru 0x90d15c8 (0x91c4 bytes) 2024-09-17T22:57:52.932 INFO:teuthology.orchestra.run.smithi159.stdout:2423 writesame 0x9a64058 thru 0x9a7359d data_size 0x46(0xf546 bytes) 2024-09-17T22:57:52.934 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:52.934 INFO:teuthology.orchestra.run.smithi159.stdout:2425 read 0x8e47932 thru 0x8e49567 (0x1c36 bytes) 2024-09-17T22:57:52.934 INFO:teuthology.orchestra.run.smithi159.stdout:2427 clone 21 order 20 su 4096 sc 7 2024-09-17T22:57:52.953 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:52.953 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:52.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:53.002 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:53.000+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:53.038 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:53.051 INFO:teuthology.orchestra.run.smithi171.stdout:get_pg_num_history pg_num_history pg_num_history(e45 pg_nums {1={15=1},2={18=8},3={22=32}} deleted_pools ) 2024-09-17T22:57:53.052 INFO:teuthology.orchestra.run.smithi171.stdout:Importing pgid 3.16 2024-09-17T22:57:53.052 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6834e080:::rbd_data.1188373e3d04.000000000000000d:6# 2024-09-17T22:57:53.056 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6834e080:::rbd_data.1188373e3d04.000000000000000d:head# 2024-09-17T22:57:53.056 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 6=[]:{6=[6]} 2024-09-17T22:57:53.057 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:68450ac3:::rbd_data.118834c4f370.0000000000000064:head# 2024-09-17T22:57:53.057 INFO:teuthology.orchestra.run.smithi171.stdout:snapset b=[]:{} 2024-09-17T22:57:53.057 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:68dcf43e:::rbd_data.118834c4f370.000000000000004a:head# 2024-09-17T22:57:53.057 INFO:teuthology.orchestra.run.smithi171.stdout:snapset b=[]:{} 2024-09-17T22:57:53.057 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:69191c60:::rbd_data.118846a2cbc0.000000000000000d:9# 2024-09-17T22:57:53.058 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:69191c60:::rbd_data.118846a2cbc0.000000000000000d:head# 2024-09-17T22:57:53.058 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 9=[]:{9=[9]} 2024-09-17T22:57:53.058 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:69bce82f:::rbd_data.1188903913c1.000000000000002c:11# 2024-09-17T22:57:53.059 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:69bce82f:::rbd_data.1188903913c1.000000000000002c:head# 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 11=[]:{11=[11]} 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6a416ce2:::rbd_object_map.1188f4a20928.0000000000000004:head# 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 0=[]:{} 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6a4cb7f8:::rbd_data.118840f89003.00000000000000bb:head# 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 5=[]:{} 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6ad3dd83:::rbd_data.118840f89003.00000000000000d4:head# 2024-09-17T22:57:53.060 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 5=[]:{} 2024-09-17T22:57:53.061 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6be9c72b:::rbd_data.118840f89003.00000000000000da:head# 2024-09-17T22:57:53.061 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 5=[]:{} 2024-09-17T22:57:53.061 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6bf17a9c:::rbd_data.118846a2cbc0.000000000000000e:9# 2024-09-17T22:57:53.061 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6bf17a9c:::rbd_data.118846a2cbc0.000000000000000e:head# 2024-09-17T22:57:53.062 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 9=[]:{9=[9]} 2024-09-17T22:57:53.062 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6de7e899:::rbd_data.118840f89003.00000000000000d7:head# 2024-09-17T22:57:53.062 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 5=[]:{} 2024-09-17T22:57:53.062 INFO:teuthology.orchestra.run.smithi171.stdout:Write #3:6e770d7a:::rbd_data.1188f4a20928.0000000000000002:head# 2024-09-17T22:57:53.069 INFO:teuthology.orchestra.run.smithi171.stdout:snapset 0=[]:{} 2024-09-17T22:57:53.096 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:53.094+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:53.096 INFO:teuthology.orchestra.run.smithi171.stdout:write_pg epoch 45 info 3.16( v 45'87 (0'0,45'87] local-lis/les=22/23 n=16 ec=22/22 lis/c=22/22 les/c/f=23/23/0 sis=22) 2024-09-17T22:57:53.097 INFO:teuthology.orchestra.run.smithi171.stdout:Import successful 2024-09-17T22:57:53.112 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:53.112 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:53.121 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:53.164 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:53.164 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:53.184 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:53.184 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:53.193 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone20 from 0xd8fbe68 (overlap 0x2679db) to 0x0 2024-09-17T22:57:53.198 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:53.195+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:53.203 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:53.219 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:53.219 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:53.235 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:53.235 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:53.236 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:53.236 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:53.236 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:53.244 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:53.254 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:53.266 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T22:57:53.272 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:53.272 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:53.275 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:53.276 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:53.277 INFO:teuthology.orchestra.run.smithi159.stdout:2428 write 0xcf36c84 thru 0xcf38e4d (0x21ca bytes) 2024-09-17T22:57:53.298 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:53.295+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:53.316 INFO:teuthology.orchestra.run.smithi159.stdout:2429 write 0x617511e thru 0x617c9a3 (0x7886 bytes) 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.508689889000003, 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:57:53.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:57:53.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:57:53.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.380 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.282006548999998, 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.381 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:57:53.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.385 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281972013000001, 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:57:53.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:57:53.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281917629999999, 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:57:53.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281892626000001, 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:57:53.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281872045, 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.400 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:57:53.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:53.404 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281850762000001, 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:57:53.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281832236, 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:57:53.409 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:57:53.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:57:53.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281814063999999, 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:57:53.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281796236000002, 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:57:53.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.281481103000001, 2024-09-17T22:57:53.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:57:53.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:57:53.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.280758832, 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:57:53.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:57:53.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.280714236000001, 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:57:53.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:57:53.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.280686914, 2024-09-17T22:57:53.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:57:53.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:57:53.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.278868116000002, 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.444 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:57:53.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.278525310999999, 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:57:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.276254178999999, 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:57:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:57:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:57:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.276222085000001, 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:57:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.276195547, 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.276176654, 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:57:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:53.472 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:53.473 INFO:teuthology.orchestra.run.smithi159.stdout:2432 punch from 0x84f659a to 0x8501225, (0xac8b bytes) 2024-09-17T22:57:53.473 INFO:teuthology.orchestra.run.smithi159.stdout:2433 trunc from 0xd8fbe68 to 0x9505a5c 2024-09-17T22:57:53.474 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:53.471+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:53.474 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:53.480 INFO:teuthology.orchestra.run.smithi159.stdout:2435 write 0x270f891 thru 0x271196e (0x20de bytes) 2024-09-17T22:57:53.513 INFO:teuthology.orchestra.run.smithi159.stdout:2436 read 0x5edf779 thru 0x5ee16ed (0x1f75 bytes) 2024-09-17T22:57:53.513 INFO:teuthology.orchestra.run.smithi159.stdout:2437 trunc from 0x9505a5c to 0x9828dc2 2024-09-17T22:57:53.525 INFO:teuthology.orchestra.run.smithi159.stdout:2438 writesame 0xb5c55fe thru 0xb5d43dd data_size 0x58(0xede0 bytes) 2024-09-17T22:57:53.525 INFO:teuthology.orchestra.run.smithi159.stdout:2439 punch from 0x10c0bba to 0x10cb942, (0xad88 bytes) 2024-09-17T22:57:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:2440 trunc from 0xb5d43de to 0x49dd6d2 2024-09-17T22:57:53.574 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:53.571+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:53.617 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /var/log/ceph/exp.3.16.5 2024-09-17T22:57:53.639 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:53.639 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:53.640 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:53.640 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:53.649 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:53.662 INFO:teuthology.orchestra.run.smithi159.stdout:2443 writesame 0x3b67a11 thru 0x3b6879a data_size 0x1(0xd8a bytes) 2024-09-17T22:57:53.695 INFO:teuthology.orchestra.run.smithi159.stdout:2445 punch from 0x253eac8 to 0x2542459, (0x3991 bytes) 2024-09-17T22:57:53.696 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:53.693+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:53.700 INFO:teuthology.orchestra.run.smithi159.stdout:2446 trunc from 0x49dd6d2 to 0x90f9882 2024-09-17T22:57:53.705 INFO:teuthology.orchestra.run.smithi159.stdout:2448 write 0x30f3505 thru 0x30f4400 (0xefc bytes) 2024-09-17T22:57:53.735 INFO:teuthology.orchestra.run.smithi159.stdout:2451 read 0x4cfa68a thru 0x4cfce41 (0x27b8 bytes) 2024-09-17T22:57:53.735 INFO:teuthology.orchestra.run.smithi159.stdout:2453 read 0x6d089fb thru 0x6d17bf9 (0xf1ff bytes) 2024-09-17T22:57:53.735 INFO:teuthology.orchestra.run.smithi159.stdout:2454 write 0x53166b1 thru 0x531a819 (0x4169 bytes) 2024-09-17T22:57:53.747 INFO:teuthology.orchestra.run.smithi159.stdout:2455 write 0xe856192 thru 0xe864f22 (0xed91 bytes) 2024-09-17T22:57:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:2456 read 0x68af20b thru 0x68b193f (0x2735 bytes) 2024-09-17T22:57:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:2458 read 0x7bd6c0d thru 0x7bd9df2 (0x31e6 bytes) 2024-09-17T22:57:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:2459 write 0x397c33 thru 0x39c969 (0x4d37 bytes) 2024-09-17T22:57:53.778 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:53.773+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:53.821 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:53.821 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:53.821 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:53.821 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:53.821 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:53.835 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:53.856 INFO:teuthology.orchestra.run.smithi159.stdout:2461 write 0xc424f9e thru 0xc434c22 (0xfc85 bytes) 2024-09-17T22:57:53.864 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:53.864 INFO:teuthology.orchestra.run.smithi159.stdout:2463 writesame 0x432574b thru 0x4325d57 data_size 0x1(0x60d bytes) 2024-09-17T22:57:53.876 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:53.874+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:53.879 INFO:teuthology.orchestra.run.smithi159.stdout:2464 read 0x9392133 thru 0x93a0222 (0xe0f0 bytes) 2024-09-17T22:57:53.879 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:53.879 INFO:teuthology.orchestra.run.smithi159.stdout:2467 write 0xd255ddc thru 0xd260da3 (0xafc8 bytes) 2024-09-17T22:57:53.887 INFO:teuthology.orchestra.run.smithi159.stdout:2468 write 0x4bcf0cb thru 0x4bd04c6 (0x13fc bytes) 2024-09-17T22:57:53.889 INFO:teuthology.orchestra.run.smithi159.stdout:2469 read 0xa3c8e84 thru 0xa3d103f (0x81bc bytes) 2024-09-17T22:57:53.889 INFO:teuthology.orchestra.run.smithi159.stdout:2470 writesame 0x7cc5e38 thru 0x7ccdbf7 data_size 0x20(0x7dc0 bytes) 2024-09-17T22:57:53.900 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:53.900 INFO:teuthology.orchestra.run.smithi159.stdout:2472 writesame 0x87907e3 thru 0x879e41b data_size 0x1(0xdc39 bytes) 2024-09-17T22:57:53.913 INFO:teuthology.orchestra.run.smithi159.stdout:2473 read 0x33c9504 thru 0x33ca222 (0xd1f bytes) 2024-09-17T22:57:53.914 INFO:teuthology.orchestra.run.smithi159.stdout:2475 punch from 0x964a486 to 0x9656bb6, (0xc730 bytes) 2024-09-17T22:57:53.914 INFO:teuthology.orchestra.run.smithi159.stdout:2476 read 0x2566d13 thru 0x2568218 (0x1506 bytes) 2024-09-17T22:57:53.914 INFO:teuthology.orchestra.run.smithi159.stdout:2478 punch from 0x7b49bb3 to 0x7b515c8, (0x7a15 bytes) 2024-09-17T22:57:53.914 INFO:teuthology.orchestra.run.smithi159.stdout:2479 writesame 0x8e1f864 thru 0x8e2a361 data_size 0x2b(0xaafe bytes) 2024-09-17T22:57:53.929 INFO:teuthology.orchestra.run.smithi159.stdout:2480 write 0x5540299 thru 0x5548524 (0x828c bytes) 2024-09-17T22:57:53.940 INFO:teuthology.orchestra.run.smithi159.stdout:2481 writesame 0x2f8d48e thru 0x2f92a11 data_size 0x1a(0x5584 bytes) 2024-09-17T22:57:53.977 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:53.974+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:53.996 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:53.996 INFO:teuthology.orchestra.run.smithi159.stdout:2483 write 0x231e2eb thru 0x2324487 (0x619d bytes) 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:54.007 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.730019878, 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:57:54.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:57:54.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729915842, 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:57:54.016 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:54.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:57:54.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.020 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729894581, 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.021 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:57:54.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729873238, 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.025 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:57:54.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:57:54.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729803781000001, 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:57:54.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.035 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729784803000001, 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.036 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:54.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.040 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729755873999999, 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.041 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.045 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729729935999998, 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.046 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:54.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:57:54.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729701941999998, 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.052 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729681018000001, 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:57:54.056 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:57:54.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729634276999999, 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.061 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:57:54.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729611868999999, 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:57:54.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:57:54.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:54.070 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729588525, 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:54.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729565835999999, 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.075 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:57:54.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:57:54.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729543636999999, 2024-09-17T22:57:54.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:57:54.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729514851000001, 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.084 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:57:54.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:57:54.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.088 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729492218000001, 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:57:54.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:57:54.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:57:54.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729472964999999, 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.096 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.097 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:57:54.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:57:54.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:57:54.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.103 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.729445789, 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:54.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:57:54.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.5518067940000009, 2024-09-17T22:57:54.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:57:54.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:57:54.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.117 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:54.120 INFO:teuthology.orchestra.run.smithi159.stdout:2485 writesame 0xe7c0ba5 thru 0xe7c8919 data_size 0x1(0x7d75 bytes) 2024-09-17T22:57:54.120 INFO:teuthology.orchestra.run.smithi159.stdout:2486 writesame 0xd56b56b thru 0xd573d17 data_size 0x1(0x87ad bytes) 2024-09-17T22:57:54.120 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.120 INFO:teuthology.orchestra.run.smithi159.stdout:2488 trunc from 0xe864f23 to 0x14d00d0 2024-09-17T22:57:54.121 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:54.116+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:54.128 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:57:54.193 INFO:teuthology.orchestra.run.smithi159.stdout:2490 writesame 0x998c3de thru 0x99947a3 data_size 0x2(0x83c6 bytes) 2024-09-17T22:57:54.194 INFO:teuthology.orchestra.run.smithi159.stdout:2491 writesame 0x529d4bc thru 0x52acd79 data_size 0x3(0xf8be bytes) 2024-09-17T22:57:54.207 INFO:teuthology.orchestra.run.smithi159.stdout:2492 punch from 0xdf760c to 0xdfb133, (0x3b27 bytes) 2024-09-17T22:57:54.207 INFO:teuthology.orchestra.run.smithi159.stdout:2493 punch from 0x65e1d89 to 0x65f0942, (0xebb9 bytes) 2024-09-17T22:57:54.209 INFO:teuthology.orchestra.run.smithi159.stdout:2494 trunc from 0x99947a4 to 0x82cb610 2024-09-17T22:57:54.215 INFO:teuthology.orchestra.run.smithi159.stdout:2495 write 0x1d73ae4 thru 0x1d7e665 (0xab82 bytes) 2024-09-17T22:57:54.219 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:54.217+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:54.223 INFO:teuthology.orchestra.run.smithi159.stdout:2497 write 0x3823097 thru 0x3827d6c (0x4cd6 bytes) 2024-09-17T22:57:54.230 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.230 INFO:teuthology.orchestra.run.smithi159.stdout:2500 read 0x50a8b22 thru 0x50a9323 (0x802 bytes) 2024-09-17T22:57:54.230 INFO:teuthology.orchestra.run.smithi159.stdout:2501 punch from 0x1bc7b00 to 0x1bcee9c, (0x739c bytes) 2024-09-17T22:57:54.231 INFO:teuthology.orchestra.run.smithi159.stdout:2503 read 0x597d3a9 thru 0x598a03b (0xcc93 bytes) 2024-09-17T22:57:54.231 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.231 INFO:teuthology.orchestra.run.smithi159.stdout:2507 read 0x799acef thru 0x799e057 (0x3369 bytes) 2024-09-17T22:57:54.231 INFO:teuthology.orchestra.run.smithi159.stdout:2508 punch from 0x6338b44 to 0x6344015, (0xb4d1 bytes) 2024-09-17T22:57:54.231 INFO:teuthology.orchestra.run.smithi159.stdout:2509 writesame 0x597cbeb thru 0x5987f07 data_size 0x1(0xb31d bytes) 2024-09-17T22:57:54.243 INFO:teuthology.orchestra.run.smithi159.stdout:2510 punch from 0x67a576 to 0x682a77, (0x8501 bytes) 2024-09-17T22:57:54.243 INFO:teuthology.orchestra.run.smithi159.stdout:2511 writesame 0x45c528e thru 0x45ccc7f data_size 0x56(0x79f2 bytes) 2024-09-17T22:57:54.257 INFO:teuthology.orchestra.run.smithi159.stdout:2512 trunc from 0x82cb610 to 0x65c6d98 2024-09-17T22:57:54.270 INFO:teuthology.orchestra.run.smithi159.stdout:2514 trunc from 0x65c6d98 to 0x1a2ee80 2024-09-17T22:57:54.287 INFO:teuthology.orchestra.run.smithi159.stdout:2517 read 0x13cacc5 thru 0x13d2b1f (0x7e5b bytes) 2024-09-17T22:57:54.288 INFO:teuthology.orchestra.run.smithi159.stdout:2518 read 0x64eec8 thru 0x652a03 (0x3b3c bytes) 2024-09-17T22:57:54.294 INFO:teuthology.orchestra.run.smithi159.stdout:2519 write 0x1b95842 thru 0x1b98610 (0x2dcf bytes) 2024-09-17T22:57:54.295 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.295 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:54.295 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:54.296 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.307 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:54.317 INFO:teuthology.orchestra.run.smithi159.stdout:2520 writesame 0x80a55a4 thru 0x80b211c data_size 0x3(0xcb79 bytes) 2024-09-17T22:57:54.320 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:54.317+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:54.331 INFO:teuthology.orchestra.run.smithi159.stdout:2521 write 0xeb7f95a thru 0xeb8118a (0x1831 bytes) 2024-09-17T22:57:54.338 INFO:teuthology.orchestra.run.smithi159.stdout:2522 read 0x7854892 thru 0x7860c24 (0xc393 bytes) 2024-09-17T22:57:54.338 INFO:teuthology.orchestra.run.smithi159.stdout:2523 write 0x16f417d thru 0x17028f3 (0xe777 bytes) 2024-09-17T22:57:54.348 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.348 INFO:teuthology.orchestra.run.smithi159.stdout:2528 writesame 0xc1e0729 thru 0xc1ef588 data_size 0x10(0xee60 bytes) 2024-09-17T22:57:54.358 INFO:teuthology.orchestra.run.smithi159.stdout:2530 read 0x90de9a1 thru 0x90e2b44 (0x41a4 bytes) 2024-09-17T22:57:54.358 INFO:teuthology.orchestra.run.smithi159.stdout:2531 writesame 0x2719a8f thru 0x2729298 data_size 0x13(0xf80a bytes) 2024-09-17T22:57:54.384 INFO:teuthology.orchestra.run.smithi159.stdout:2532 trunc from 0xeb8118b to 0x5697f7 2024-09-17T22:57:54.420 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.420 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.420 INFO:teuthology.orchestra.run.smithi159.stdout:2538 punch from 0x162c7e to 0x16a6e7, (0x7a69 bytes) 2024-09-17T22:57:54.421 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.421 INFO:teuthology.orchestra.run.smithi159.stdout:2540 read 0xc7c65 thru 0xce15b (0x64f7 bytes) 2024-09-17T22:57:54.421 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:54.418+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:54.431 INFO:teuthology.orchestra.run.smithi159.stdout:2541 trunc from 0x5697f7 to 0x86ea180 2024-09-17T22:57:54.436 INFO:teuthology.orchestra.run.smithi159.stdout:2542 writesame 0x80c17ed thru 0x80cfbfd data_size 0x1(0xe411 bytes) 2024-09-17T22:57:54.444 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.444 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:54.444 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:54.444 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:54.444 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.451 INFO:teuthology.orchestra.run.smithi159.stdout:2544 write 0x6ae983f thru 0x6af3a1f (0xa1e1 bytes) 2024-09-17T22:57:54.453 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:54.468 INFO:teuthology.orchestra.run.smithi159.stdout:2545 writesame 0x91e4e8b thru 0x91edf5b data_size 0x1(0x90d1 bytes) 2024-09-17T22:57:54.468 INFO:teuthology.orchestra.run.smithi159.stdout:2546 punch from 0x362a28a to 0x362f17b, (0x4ef1 bytes) 2024-09-17T22:57:54.469 INFO:teuthology.orchestra.run.smithi159.stdout:2547 writesame 0x530234a thru 0x5303c63 data_size 0x2(0x191a bytes) 2024-09-17T22:57:54.474 INFO:teuthology.orchestra.run.smithi159.stdout:2548 read 0x28c8c99 thru 0x28d35d1 (0xa939 bytes) 2024-09-17T22:57:54.474 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.487 INFO:teuthology.orchestra.run.smithi159.stdout:2551 write 0xd793998 thru 0xd7a2041 (0xe6aa bytes) 2024-09-17T22:57:54.496 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.499 INFO:teuthology.orchestra.run.smithi159.stdout:2556 trunc from 0xd7a2042 to 0xac2149a 2024-09-17T22:57:54.509 INFO:teuthology.orchestra.run.smithi159.stdout:2557 punch from 0x992fc52 to 0x99360ae, (0x645c bytes) 2024-09-17T22:57:54.509 INFO:teuthology.orchestra.run.smithi159.stdout:2559 write 0x6fc6689 thru 0x6fd2884 (0xc1fc bytes) 2024-09-17T22:57:54.512 INFO:teuthology.orchestra.run.smithi159.stdout:2560 read 0xa9740fc thru 0xa97a6fe (0x6603 bytes) 2024-09-17T22:57:54.512 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.519 INFO:teuthology.orchestra.run.smithi159.stdout:2563 write 0xcf6ef23 thru 0xcf71747 (0x2825 bytes) 2024-09-17T22:57:54.520 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:54.518+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:2564 punch from 0x71a7563 to 0x71a9bd7, (0x2674 bytes) 2024-09-17T22:57:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:2566 punch from 0x1d51ff9 to 0x1d57cfb, (0x5d02 bytes) 2024-09-17T22:57:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:2567 read 0x3829524 thru 0x383181f (0x82fc bytes) 2024-09-17T22:57:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:2568 read 0x8b6dde7 thru 0x8b76077 (0x8291 bytes) 2024-09-17T22:57:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:2570 punch from 0x9753fe8 to 0x975f998, (0xb9b0 bytes) 2024-09-17T22:57:54.529 INFO:teuthology.orchestra.run.smithi159.stdout:2571 read 0x7d8858d thru 0x7d89b38 (0x15ac bytes) 2024-09-17T22:57:54.529 INFO:teuthology.orchestra.run.smithi159.stdout:2572 write 0x67874ec thru 0x6796d72 (0xf887 bytes) 2024-09-17T22:57:54.538 INFO:teuthology.orchestra.run.smithi159.stdout:2573 write 0x5dd726b thru 0x5de61da (0xef70 bytes) 2024-09-17T22:57:54.546 INFO:teuthology.orchestra.run.smithi159.stdout:2574 read 0x6f208a2 thru 0x6f22266 (0x19c5 bytes) 2024-09-17T22:57:54.546 INFO:teuthology.orchestra.run.smithi159.stdout:2575 read 0x52565fb thru 0x525a355 (0x3d5b bytes) 2024-09-17T22:57:54.547 INFO:teuthology.orchestra.run.smithi159.stdout:2576 write 0x12e6773 thru 0x12ee2e6 (0x7b74 bytes) 2024-09-17T22:57:54.555 INFO:teuthology.orchestra.run.smithi159.stdout:2577 punch from 0xae51929 to 0xae608d1, (0xefa8 bytes) 2024-09-17T22:57:54.555 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.555 INFO:teuthology.orchestra.run.smithi159.stdout:2579 write 0x1b10b4 thru 0x1bfa35 (0xe982 bytes) 2024-09-17T22:57:54.558 INFO:teuthology.orchestra.run.smithi159.stdout:2580 read 0x60df0f2 thru 0x60e66d9 (0x75e8 bytes) 2024-09-17T22:57:54.558 INFO:teuthology.orchestra.run.smithi159.stdout:2581 punch from 0xa29a10f to 0xa2a738f, (0xd280 bytes) 2024-09-17T22:57:54.559 INFO:teuthology.orchestra.run.smithi159.stdout:2582 read 0x7ba8b6d thru 0x7bb5cbe (0xd152 bytes) 2024-09-17T22:57:54.559 INFO:teuthology.orchestra.run.smithi159.stdout:2583 read 0xa340eff thru 0xa3500ac (0xf1ae bytes) 2024-09-17T22:57:54.559 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.559 INFO:teuthology.orchestra.run.smithi159.stdout:2585 write 0x38d9b2f thru 0x38e908b (0xf55d bytes) 2024-09-17T22:57:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:2586 write 0xa864518 thru 0xa86f074 (0xab5d bytes) 2024-09-17T22:57:54.580 INFO:teuthology.orchestra.run.smithi159.stdout:2587 read 0x629ad50 thru 0x629da1d (0x2cce bytes) 2024-09-17T22:57:54.580 INFO:teuthology.orchestra.run.smithi159.stdout:2588 read 0xcec6750 thru 0xcecdd60 (0x7611 bytes) 2024-09-17T22:57:54.584 INFO:teuthology.orchestra.run.smithi159.stdout:2590 trunc from 0xcf71748 to 0xa301e45 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16134 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329647703999999, 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036277120000000003, 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16134 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255129+0000", 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255131+0000", 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.916e-06 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255137+0000", 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3509999999999998e-06 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255142+0000", 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.882e-06 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291253+0000", 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036111153 2024-09-17T22:57:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291337+0000", 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3535000000000006e-05 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291405+0000", 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8858999999999999e-05 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16135 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329627730999999, 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036429299999999998, 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16135 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255149+0000", 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255151+0000", 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255157+0000", 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255162+0000", 2024-09-17T22:57:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291437+0000", 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036274726 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291514+0000", 2024-09-17T22:57:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7342000000000006e-05 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291578+0000", 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3634999999999996e-05 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16159 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329607992, 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036685328000000003, 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16159 2024-09-17T22:57:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255170+0000", 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255177+0000", 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4989999999999999e-06 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255193+0000", 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909999999999998e-05 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291665+0000", 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036471816999999997 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291751+0000", 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6509999999999997e-05 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291853+0000", 2024-09-17T22:57:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102099 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16161 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329570441000001, 2024-09-17T22:57:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036865504, 2024-09-17T22:57:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16161 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255208+0000", 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3250000000000002e-06 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255218+0000", 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4080000000000004e-06 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255223+0000", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291884+0000", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036661457000000001 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291966+0000", 2024-09-17T22:57:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1920000000000002e-05 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292071+0000", 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000104608 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16163 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329545991, 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037090170999999998, 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16163 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:57:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255233+0000", 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255242+0000", 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0750000000000004e-06 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255247+0000", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292101+0000", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036854162000000003 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292174+0000", 2024-09-17T22:57:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2805000000000005e-05 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292320+0000", 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014646799999999999 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16165 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1728512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329522553, 2024-09-17T22:57:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037305086000000001, 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16165 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:54.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255254+0000", 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255256+0000", 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.12e-06 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255265+0000", 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8959999999999992e-06 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255270+0000", 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8760000000000004e-06 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292365+0000", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037095377999999998 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292452+0000", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6947000000000006e-05 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292558+0000", 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010643 2024-09-17T22:57:54.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16167 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329499516999999, 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037507133999999998, 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.623 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16167 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255277+0000", 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255279+0000", 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255288+0000", 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8429999999999999e-06 2024-09-17T22:57:54.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255293+0000", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292603+0000", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037309869000000002 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292681+0000", 2024-09-17T22:57:54.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8484000000000003e-05 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292784+0000", 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010224 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16169 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329476496000002, 2024-09-17T22:57:54.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037775437000000002, 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16169 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:54.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255300+0000", 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255302+0000", 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255308+0000", 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.631 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255313+0000", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292863+0000", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037549224999999999 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292969+0000", 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010604500000000001 2024-09-17T22:57:54.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293075+0000", 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010620699999999999 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16171 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1875968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329456602, 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037958219000000001, 2024-09-17T22:57:54.633 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16171 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255320+0000", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2899999999999999e-07 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255322+0000", 2024-09-17T22:57:54.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.906e-06 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255337+0000", 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5452999999999999e-05 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255343+0000", 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293106+0000", 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037763707000000001 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293186+0000", 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0153000000000003e-05 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293278+0000", 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1114999999999998e-05 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.637 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16172 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1925120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329426319, 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038145196999999999, 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.638 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16172 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8400000000000004e-07 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255359+0000", 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:57:54.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255367+0000", 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0649999999999999e-06 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255372+0000", 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1390000000000003e-06 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293318+0000", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037945566 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293399+0000", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1168999999999995e-05 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293495+0000", 2024-09-17T22:57:54.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6162000000000005e-05 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16173 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329395124000001, 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038380816999999998, 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.643 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16173 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255388+0000", 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255402+0000", 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4532000000000001e-05 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255408+0000", 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7740000000000001e-06 2024-09-17T22:57:54.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293577+0000", 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038169526000000002 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293664+0000", 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6312999999999998e-05 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293762+0000", 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7947000000000003e-05 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.648 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16174 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329360368, 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038531986999999997, 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16174 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.649 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:57:54.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255418+0000", 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255426+0000", 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1620000000000002e-06 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255436+0000", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0851e-05 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293804+0000", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038367215000000003 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293886+0000", 2024-09-17T22:57:54.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2189999999999997e-05 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293948+0000", 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1709000000000002e-05 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16176 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:54.653 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.329332415, 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038738529000000001, 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16176 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:54.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255446+0000", 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255453+0000", 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8170000000000001e-06 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255458+0000", 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293991+0000", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038532376 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294084+0000", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3614000000000002e-05 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294182+0000", 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7949999999999996e-05 2024-09-17T22:57:54.657 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16239 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325959955999998, 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036338338999999997, 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16239 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258819+0000", 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3e-06 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258825+0000", 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258830+0000", 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294968+0000", 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036137423000000002 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295061+0000", 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3177999999999995e-05 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295154+0000", 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3645000000000002e-05 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.662 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16241 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1835008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325935390000001, 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036566098999999998, 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.663 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16241 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4770000000000002e-06 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258850+0000", 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5490000000000003e-06 2024-09-17T22:57:54.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258855+0000", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.904e-06 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295247+0000", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036392378000000003 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295334+0000", 2024-09-17T22:57:54.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7058999999999994e-05 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295407+0000", 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2287999999999996e-05 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16243 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325913525000001, 2024-09-17T22:57:54.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036705072999999998, 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16243 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258863+0000", 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258866+0000", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.678e-06 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258872+0000", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258877+0000", 2024-09-17T22:57:54.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295445+0000", 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036567582000000001 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295506+0000", 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1895000000000003e-05 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295568+0000", 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1085000000000002e-05 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16245 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1933312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325586939000001, 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036671423000000002, 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.672 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16245 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259190+0000", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259194+0000", 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259205+0000", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0468e-05 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259212+0000", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1380000000000001e-06 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295666+0000", 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036453700999999998 2024-09-17T22:57:54.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295753+0000", 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7321e-05 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295860+0000", 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010761599999999999 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16246 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325552382000001, 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036882379, 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16246 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.677 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:57:54.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6599999999999996e-07 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259227+0000", 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259234+0000", 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8539999999999999e-06 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259239+0000", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295903+0000", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036664018999999999 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295993+0000", 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9944000000000005e-05 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296106+0000", 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113158 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16248 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325530212, 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037070651000000003, 2024-09-17T22:57:54.681 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16248 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259248+0000", 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:57:54.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259254+0000", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259260+0000", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4489999999999998e-06 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296156+0000", 2024-09-17T22:57:54.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036896089999999999 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296238+0000", 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2616999999999999e-05 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296316+0000", 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7964000000000001e-05 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16249 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.325359324000001, 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037100148999999999, 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:54.686 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16249 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:54.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8700000000000005e-07 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259421+0000", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0470000000000004e-06 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259431+0000", 2024-09-17T22:57:54.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4190000000000008e-06 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259437+0000", 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4520000000000002e-06 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296352+0000", 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036914354000000003 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296427+0000", 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5603000000000001e-05 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296517+0000", 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9586999999999995e-05 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.690 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:54.691 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:54.691 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.692 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.692 INFO:teuthology.orchestra.run.smithi159.stdout:2592 write 0x38b051d thru 0x38c0145 (0xfc29 bytes) 2024-09-17T22:57:54.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:54.696 INFO:teuthology.orchestra.run.smithi159.stdout:2593 trunc from 0xa301e45 to 0xe25f455 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2594 punch from 0xd089310 to 0xd092abf, (0x97af bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2596 write 0xcfdfe4 thru 0xd060c7 (0x80e4 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2598 write 0xcaa64a thru 0xcad18b (0x2b42 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2599 write 0xb8efb0e thru 0xb8f2920 (0x2e13 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2601 trunc from 0xe25f455 to 0xdea3d93 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2602 read 0xac3b8e4 thru 0xac41d3b (0x6458 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2605 write 0x47ed24f thru 0x47faf05 (0xdcb7 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2606 write 0xbc4ef7e thru 0xbc5d8d7 (0xe95a bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2607 read 0x85da0a6 thru 0x85e78ce (0xd829 bytes) 2024-09-17T22:57:54.697 INFO:teuthology.orchestra.run.smithi159.stdout:2608 write 0x660ae2f thru 0x6619a5a (0xec2c bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2610 writesame 0xeaafee7 thru 0xeabeb71 data_size 0xb(0xec8b bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2612 writesame 0xcc58ab7 thru 0xcc5bb2d data_size 0x1(0x3077 bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2613 punch from 0x95d556b to 0x95e064f, (0xb0e4 bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2614 writesame 0xb15fa04 thru 0xb161e4c data_size 0x1(0x2449 bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2615 read 0xd045e27 thru 0xd055a20 (0xfbfa bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2616 write 0xe20ade0 thru 0xe20fb7a (0x4d9b bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2618 punch from 0x2aa319f to 0x2aae6ff, (0xb560 bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2620 writesame 0x2c1c738 thru 0x2c1d016 data_size 0x1(0x8df bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2621 punch from 0x3d5d93e to 0x3d68cda, (0xb39c bytes) 2024-09-17T22:57:54.698 INFO:teuthology.orchestra.run.smithi159.stdout:2624 write 0xe8504ae thru 0xe854020 (0x3b73 bytes) 2024-09-17T22:57:54.699 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:54.689+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:54.699 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.699 INFO:teuthology.orchestra.run.smithi159.stdout:2626 writesame 0xd4de270 thru 0xd4ed436 data_size 0x1(0xf1c7 bytes) 2024-09-17T22:57:54.710 INFO:teuthology.orchestra.run.smithi159.stdout:2627 punch from 0x292216b to 0x2931504, (0xf399 bytes) 2024-09-17T22:57:54.710 INFO:teuthology.orchestra.run.smithi159.stdout:2630 punch from 0x46e5ec2 to 0x46ed656, (0x7794 bytes) 2024-09-17T22:57:54.711 INFO:teuthology.orchestra.run.smithi159.stdout:2631 write 0xa7a05b4 thru 0xa7a47b3 (0x4200 bytes) 2024-09-17T22:57:54.720 INFO:teuthology.orchestra.run.smithi159.stdout:2632 punch from 0x6916f8d to 0x691b23f, (0x42b2 bytes) 2024-09-17T22:57:54.720 INFO:teuthology.orchestra.run.smithi159.stdout:2633 write 0xd3c914a thru 0xd3d64c9 (0xd380 bytes) 2024-09-17T22:57:54.722 INFO:teuthology.orchestra.run.smithi159.stdout:2634 write 0x4870d23 thru 0x4877b37 (0x6e15 bytes) 2024-09-17T22:57:54.724 INFO:teuthology.orchestra.run.smithi159.stdout:2635 read 0x7e60388 thru 0x7e68a8d (0x8706 bytes) 2024-09-17T22:57:54.725 INFO:teuthology.orchestra.run.smithi159.stdout:2636 read 0x45cb4a2 thru 0x45d2afa (0x7659 bytes) 2024-09-17T22:57:54.725 INFO:teuthology.orchestra.run.smithi159.stdout:2638 read 0x49c7a3e thru 0x49d6367 (0xe92a bytes) 2024-09-17T22:57:54.725 INFO:teuthology.orchestra.run.smithi159.stdout:2640 punch from 0xcbb4fb8 to 0xcbb9880, (0x48c8 bytes) 2024-09-17T22:57:54.726 INFO:teuthology.orchestra.run.smithi159.stdout:2641 write 0xd84d9ee thru 0xd85a8c8 (0xcedb bytes) 2024-09-17T22:57:54.727 INFO:teuthology.orchestra.run.smithi159.stdout:2642 read 0xc261e15 thru 0xc26fc45 (0xde31 bytes) 2024-09-17T22:57:54.732 INFO:teuthology.orchestra.run.smithi159.stdout:2643 trunc from 0xeabeb72 to 0xabc56c3 2024-09-17T22:57:54.748 INFO:teuthology.orchestra.run.smithi159.stdout:2645 read 0x5daaf92 thru 0x5dabd93 (0xe02 bytes) 2024-09-17T22:57:54.748 INFO:teuthology.orchestra.run.smithi159.stdout:2646 write 0x3bb6c27 thru 0x3bbc626 (0x5a00 bytes) 2024-09-17T22:57:54.751 INFO:teuthology.orchestra.run.smithi159.stdout:2647 read 0x54031c1 thru 0x540732f (0x416f bytes) 2024-09-17T22:57:54.751 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.751 INFO:teuthology.orchestra.run.smithi159.stdout:2651 read 0x9ec67f0 thru 0x9ec9110 (0x2921 bytes) 2024-09-17T22:57:54.751 INFO:teuthology.orchestra.run.smithi159.stdout:2653 punch from 0x509faf to 0x50dcc3, (0x3d14 bytes) 2024-09-17T22:57:54.755 INFO:teuthology.orchestra.run.smithi159.stdout:2654 trunc from 0xabc56c3 to 0x76fe5b6 2024-09-17T22:57:54.765 INFO:teuthology.orchestra.run.smithi159.stdout:2655 punch from 0x4725503 to 0x4726a99, (0x1596 bytes) 2024-09-17T22:57:54.771 INFO:teuthology.orchestra.run.smithi159.stdout:2656 writesame 0x8c8bafd thru 0x8c8d861 data_size 0x23(0x1d65 bytes) 2024-09-17T22:57:54.772 INFO:teuthology.orchestra.run.smithi159.stdout:2657 write 0x1c6c50f thru 0x1c6ec69 (0x275b bytes) 2024-09-17T22:57:54.776 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.777 INFO:teuthology.orchestra.run.smithi159.stdout:2659 trunc from 0x8c8d862 to 0x75c9d4d 2024-09-17T22:57:54.783 INFO:teuthology.orchestra.run.smithi159.stdout:2661 write 0x116f4a1 thru 0x117d006 (0xdb66 bytes) 2024-09-17T22:57:54.785 INFO:teuthology.orchestra.run.smithi159.stdout:2664 read 0x597d296 thru 0x598bcf6 (0xea61 bytes) 2024-09-17T22:57:54.790 INFO:teuthology.orchestra.run.smithi159.stdout:2665 trunc from 0x75c9d4d to 0xb1aff2c 2024-09-17T22:57:54.793 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:54.791+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:54.795 INFO:teuthology.orchestra.run.smithi159.stdout:2666 read 0x85f087b thru 0x85f97ce (0x8f54 bytes) 2024-09-17T22:57:54.795 INFO:teuthology.orchestra.run.smithi159.stdout:2669 write 0x54f91dc thru 0x54fd1e7 (0x400c bytes) 2024-09-17T22:57:54.803 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:54.803 INFO:teuthology.orchestra.run.smithi159.stdout:2671 writesame 0x3242a89 thru 0x324fdf5 data_size 0x19(0xd36d bytes) 2024-09-17T22:57:54.823 INFO:teuthology.orchestra.run.smithi159.stdout:2672 writesame 0xc639dbf thru 0xc64783f data_size 0x3d(0xda81 bytes) 2024-09-17T22:57:54.832 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.832 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:54.832 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:54.832 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.835 INFO:teuthology.orchestra.run.smithi159.stdout:2673 trunc from 0xc647840 to 0x3212c9c 2024-09-17T22:57:54.846 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:54.853 INFO:teuthology.orchestra.run.smithi159.stdout:2674 read 0x2fa453d thru 0x2fa8679 (0x413d bytes) 2024-09-17T22:57:54.853 INFO:teuthology.orchestra.run.smithi159.stdout:2675 punch from 0x511461 to 0x5212f3, (0xfe92 bytes) 2024-09-17T22:57:54.862 INFO:teuthology.orchestra.run.smithi159.stdout:2676 writesame 0x5b312e9 thru 0x5b37d99 data_size 0xd(0x6ab1 bytes) 2024-09-17T22:57:54.862 INFO:teuthology.orchestra.run.smithi159.stdout:2677 read 0x220c701 thru 0x221b8b2 (0xf1b2 bytes) 2024-09-17T22:57:54.862 INFO:teuthology.orchestra.run.smithi159.stdout:2678 clone 22 order 21 su 1048576 sc 13 2024-09-17T22:57:54.894 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:54.891+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:54.956 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:54.956 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:54.956 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:54.956 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:54.957 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:54.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:54.996 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:54.993+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:55.096 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:55.094+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:55.109 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 45.215604183000004, 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1044 3.9 e28/22 3:954c0790:::rbd_data.118840f89003.0000000000000090:head v 28'4, mlcod=25'2)", 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.077802478999999, 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032794199999999999, 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.09e-07 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028800+0000", 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7649999999999999e-05 2024-09-17T22:57:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028812+0000", 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1865000000000001e-05 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028819+0000", 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7750000000000004e-06 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028870+0000", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0949000000000003e-05 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028889+0000", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9199999999999999e-05 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032019+0000", 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031302230000000001 2024-09-17T22:57:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032061+0000", 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1949000000000001e-05 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 41 entries 28'5 (0'0) error 3:9656af7d:::rbd_data.118840f89003.0000000000000092:head by client.4488.0:1048 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.077729332000001, 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034247520000000001, 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028863+0000", 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7249999999999997e-06 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028879+0000", 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5862000000000002e-05 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028892+0000", 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3076e-05 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.029115+0000", 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000223427 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032279+0000", 2024-09-17T22:57:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.003164189 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.4 epoch 28/22 rep_tid 35 entries 28'5 (0'0) error 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head by client.4488.0:1103 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'3)", 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.068020500999999, 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00299241, 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038564+0000", 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6199999999999997e-07 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038581+0000", 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7116e-05 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038603+0000", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.177e-05 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038623+0000", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9772e-05 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.038712+0000", 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8910000000000001e-05 2024-09-17T22:57:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.041556+0000", 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028440800000000001 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 50 entries 28'7 (0'0) error 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head by client.4488.0:1110 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.067127205999999, 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0077516210000000002, 2024-09-17T22:57:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039458+0000", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.99e-07 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039466+0000", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4419999999999998e-06 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039486+0000", 2024-09-17T22:57:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0523999999999999e-05 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039502+0000", 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5930000000000002e-05 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039590+0000", 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8012999999999999e-05 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047208+0000", 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0076180129999999999 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1121 3.4 e28/22 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head v 28'6, mlcod=27'4)", 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.064024500999999, 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004866296, 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042566+0000", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042575+0000", 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3349999999999996e-06 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042580+0000", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042623+0000", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3396000000000003e-05 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042644+0000", 2024-09-17T22:57:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1155999999999999e-05 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047379+0000", 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047352699999999998 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047426+0000", 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6424999999999997e-05 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6635 3.4 e32/22 3:264a9651:::rbd_data.118846a2cbc0.0000000000000014:head v 32'17, mlcod=32'15)", 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 24.173416636999999, 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031534419999999998, 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933167+0000", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933168+0000", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933175+0000", 2024-09-17T22:57:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8179999999999998e-06 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933185+0000", 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0155e-05 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933191+0000", 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0499999999999997e-06 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933235+0000", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3951e-05 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.933250+0000", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4605e-05 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936290+0000", 2024-09-17T22:57:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030402879999999999 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.936321+0000", 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0905999999999999e-05 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6774 3.0 e33/22 3:05c44946:::rbd_data.118846a2cbc0.0000000000000045:head v 33'36, mlcod=32'33)", 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 24.031139454000002, 2024-09-17T22:57:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053547780000000001, 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0299999999999998e-07 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075451+0000", 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075468+0000", 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7212999999999999e-05 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075474+0000", 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075522+0000", 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8114e-05 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075543+0000", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0389000000000001e-05 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080753+0000", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052103440000000004 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080799+0000", 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6335999999999998e-05 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7337 3.11 e33/28 3:89c4a1e2:::rbd_header.118846a2cbc0:head v 33'39, mlcod=33'36)", 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.911295841000001, 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046186769999999999, 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195289+0000", 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195293+0000", 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.155e-06 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195301+0000", 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8520000000000004e-06 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195307+0000", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195340+0000", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3090999999999999e-05 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195353+0000", 2024-09-17T22:57:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2757e-05 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199876+0000", 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045229500000000004 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199907+0000", 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1340000000000001e-05 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22674 3.0 e45/22 3:000cde75:::rbd_data.1188903913c1.000000000000001f:head v 45'77, mlcod=44'74)", 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.966640423999999, 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0062289370000000004, 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139953+0000", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0229999999999998e-06 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139962+0000", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-06 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139968+0000", 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:57:55.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140030+0000", 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1803000000000004e-05 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140046+0000", 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6093999999999999e-05 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146129+0000", 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0060834239999999996 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146173+0000", 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3238999999999997e-05 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.0 epoch 45/22 rep_tid 301 entries 45'78 (0'0) error 3:047d1ef5:::rbd_data.1188903913c1.0000000000000029:head by client.4488.0:22684 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 44'74)", 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:55.155 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.964942186, 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.007471218, 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.44e-07 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141651+0000", 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4449999999999996e-06 2024-09-17T22:57:55.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141671+0000", 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0373e-05 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141695+0000", 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4224e-05 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141782+0000", 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6514999999999994e-05 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149113+0000", 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0073311169999999998 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22687 3.4 e45/22 3:25296614:::rbd_data.1188903913c1.000000000000002b:head v 45'156, mlcod=44'152)", 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:55.159 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.964068572, 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0066082959999999996, 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:55.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142516+0000", 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3900000000000004e-07 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142523+0000", 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7429999999999996e-06 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142536+0000", 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3151e-05 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142543+0000", 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7100000000000001e-06 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142607+0000", 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3765999999999999e-05 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142627+0000", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.035e-05 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149088+0000", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0064607140000000002 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149124+0000", 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6123e-05 2024-09-17T22:57:55.163 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22907 3.4 e45/22 3:244fb18a:::rbd_data.1188903913c1.00000000000001f6:head v 45'158, mlcod=45'154)", 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.947495658999999, 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031838320000000002, 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.164 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159089+0000", 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5499999999999998e-07 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159095+0000", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4409999999999998e-06 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159108+0000", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.258e-05 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159115+0000", 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9419999999999998e-06 2024-09-17T22:57:55.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159210+0000", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4853999999999997e-05 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159231+0000", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1583e-05 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162234+0000", 2024-09-17T22:57:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030021539999999999 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162272+0000", 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8622999999999999e-05 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:39761 3.4 e54/22 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:head v 54'167, mlcod=54'165)", 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:55.168 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.2875521839999999, 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031656729999999999, 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819033+0000", 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.562e-06 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819756+0000", 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00072241600000000003 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819777+0000", 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1429000000000001e-05 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819787+0000", 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0137e-05 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819890+0000", 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102441 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819913+0000", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3529000000000001e-05 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822139+0000", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022256530000000002 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822198+0000", 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8505999999999999e-05 2024-09-17T22:57:55.172 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:42197 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.0611405300000001, 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032112489999999998, 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:55.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 42197 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:55.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045445+0000", 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.615e-06 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045452+0000", 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8589999999999999e-06 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045463+0000", 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1001e-05 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045481+0000", 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8423e-05 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045537+0000", 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6069999999999997e-05 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045626+0000", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8351999999999997e-05 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048535+0000", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029092060000000001 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048589+0000", 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3761999999999998e-05 2024-09-17T22:57:55.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048655+0000", 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5961000000000001e-05 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44169 3.11 3:8b5f814e:::rbd_data.118894cf7479.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.87069643100000005, 2024-09-17T22:57:55.178 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0042738230000000004, 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44169 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235890+0000", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0789999999999999e-06 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235900+0000", 2024-09-17T22:57:55.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0277e-05 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235923+0000", 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2693000000000001e-05 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235933+0000", 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0441e-05 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236005+0000", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1957000000000002e-05 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236091+0000", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5718999999999997e-05 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240059+0000", 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0039685099999999997 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240096+0000", 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6431999999999999e-05 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240161+0000", 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5715000000000004e-05 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.183 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44203 3.0 e54/22 3:037ebc53:::rbd_data.118894cf7479.000000000000003a:head v 54'130, mlcod=54'128)", 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.83462029699999996, 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0035646229999999998, 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271965+0000", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1739999999999999e-06 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271973+0000", 2024-09-17T22:57:55.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8650000000000001e-06 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271986+0000", 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2899e-05 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271993+0000", 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2520000000000002e-06 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272061+0000", 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392000000000005e-05 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272083+0000", 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2161e-05 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275498+0000", 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034148059999999998 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275528+0000", 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0074e-05 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44204 3.4 e54/22 3:23e33dd1:::rbd_data.118894cf7479.000000000000003b:head v 54'172, mlcod=54'170)", 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.83455509500000002, 2024-09-17T22:57:55.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034598849999999998, 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:55.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.01e-07 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272034+0000", 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6179999999999997e-06 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272041+0000", 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4290000000000004e-06 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272047+0000", 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3890000000000004e-06 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272096+0000", 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9033000000000003e-05 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.191 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272114+0000", 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8158000000000001e-05 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275444+0000", 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033300489999999999 2024-09-17T22:57:55.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275489+0000", 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4808000000000002e-05 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44658 3.11 3:8a68e66c:::rbd_data.118894cf7479.000000000000007f:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.66826191400000001, 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033015309999999999, 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44658 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438324+0000", 2024-09-17T22:57:55.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.153e-06 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438332+0000", 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7759999999999997e-06 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438349+0000", 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7005999999999999e-05 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438359+0000", 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0186999999999999e-05 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438420+0000", 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0915000000000002e-05 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.197 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438517+0000", 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6848999999999994e-05 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441513+0000", 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002996383 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441545+0000", 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1656999999999999e-05 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441624+0000", 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8604999999999997e-05 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:45020 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.41240129399999997, 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0036221339999999999, 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 45020 2024-09-17T22:57:55.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694185+0000", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694195+0000", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0603e-05 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694222+0000", 2024-09-17T22:57:55.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6573000000000001e-05 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694236+0000", 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3816e-05 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694368+0000", 2024-09-17T22:57:55.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013227200000000001 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694508+0000", 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014027700000000001 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697713+0000", 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032041769999999999 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697749+0000", 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6742999999999997e-05 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697805+0000", 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5535999999999998e-05 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.206 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:55.206 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:55.206 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:55.207 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:55.208 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:57:55.209 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:55.204+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:55.209 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone21 from 0x5b37d9a (overlap 0x5697f7) to 0x0 2024-09-17T22:57:55.277 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:55.290 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T22:57:55.291 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.291 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.297 INFO:teuthology.orchestra.run.smithi159.stdout:2682 trunc from 0x5b37d9a to 0x2d50a6 2024-09-17T22:57:55.307 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:55.305+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:55.334 INFO:teuthology.orchestra.run.smithi159.stdout:2683 writesame 0x55d62e0 thru 0x55d76ca data_size 0x1(0x13eb bytes) 2024-09-17T22:57:55.334 INFO:teuthology.orchestra.run.smithi159.stdout:2685 write 0x4934294 thru 0x494217f (0xdeec bytes) 2024-09-17T22:57:55.343 INFO:teuthology.orchestra.run.smithi159.stdout:2687 write 0x5eb3373 thru 0x5ebbed4 (0x8b62 bytes) 2024-09-17T22:57:55.346 INFO:teuthology.orchestra.run.smithi159.stdout:2688 read 0x4adfdf2 thru 0x4ae6d42 (0x6f51 bytes) 2024-09-17T22:57:55.346 INFO:teuthology.orchestra.run.smithi159.stdout:2689 write 0x858358 thru 0x85b72e (0x33d7 bytes) 2024-09-17T22:57:55.356 INFO:teuthology.orchestra.run.smithi159.stdout:2692 trunc from 0x5ebbed5 to 0x1eeb8b 2024-09-17T22:57:55.366 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:55.367 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:55.367 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:55.367 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:55.371 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.371 INFO:teuthology.orchestra.run.smithi159.stdout:2697 punch from 0xfb347 to 0x109a79, (0xe732 bytes) 2024-09-17T22:57:55.380 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:55.387 INFO:teuthology.orchestra.run.smithi159.stdout:2698 writesame 0x891fd83 thru 0x8922223 data_size 0x1(0x24a1 bytes) 2024-09-17T22:57:55.399 INFO:teuthology.orchestra.run.smithi159.stdout:2699 write 0xe7c43b7 thru 0xe7cca2d (0x8677 bytes) 2024-09-17T22:57:55.403 INFO:teuthology.orchestra.run.smithi159.stdout:2700 writesame 0x1a81f5a thru 0x1a8ea73 data_size 0x1(0xcb1a bytes) 2024-09-17T22:57:55.408 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:55.406+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:55.430 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:55.430 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:55.432 INFO:teuthology.orchestra.run.smithi159.stdout:2701 write 0x882ed02 thru 0x883598f (0x6c8e bytes) 2024-09-17T22:57:55.436 INFO:teuthology.orchestra.run.smithi159.stdout:2702 writesame 0x91efe68 thru 0x91f09ad data_size 0xd(0xb46 bytes) 2024-09-17T22:57:55.439 INFO:teuthology.orchestra.run.smithi159.stdout:2703 write 0x69ef501 thru 0x69f8f18 (0x9a18 bytes) 2024-09-17T22:57:55.442 INFO:teuthology.orchestra.run.smithi159.stdout:2704 read 0xf36992 thru 0xf43b17 (0xd186 bytes) 2024-09-17T22:57:55.442 INFO:teuthology.orchestra.run.smithi159.stdout:2705 punch from 0x957385f to 0x9576c95, (0x3436 bytes) 2024-09-17T22:57:55.442 INFO:teuthology.orchestra.run.smithi159.stdout:2708 write 0x2d63805 thru 0x2d69a8a (0x6286 bytes) 2024-09-17T22:57:55.444 INFO:teuthology.orchestra.run.smithi159.stdout:2709 write 0xcd6090b thru 0xcd62cca (0x23c0 bytes) 2024-09-17T22:57:55.449 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:55.449 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:55.450 INFO:teuthology.orchestra.run.smithi159.stdout:2711 read 0x5c1613c thru 0x5c18613 (0x24d8 bytes) 2024-09-17T22:57:55.450 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.450 INFO:teuthology.orchestra.run.smithi159.stdout:2716 punch from 0xc2ed5eb to 0xc2fb05f, (0xda74 bytes) 2024-09-17T22:57:55.450 INFO:teuthology.orchestra.run.smithi159.stdout:2717 writesame 0x6d50f4b thru 0x6d5a4a7 data_size 0x1(0x955d bytes) 2024-09-17T22:57:55.465 INFO:teuthology.orchestra.run.smithi159.stdout:2719 read 0x5f0ef83 thru 0x5f184c1 (0x953f bytes) 2024-09-17T22:57:55.465 INFO:teuthology.orchestra.run.smithi159.stdout:2721 read 0x2a7de5 thru 0x2ae66f (0x688b bytes) 2024-09-17T22:57:55.465 INFO:teuthology.orchestra.run.smithi159.stdout:2722 writesame 0xd3162a4 thru 0xd31c3de data_size 0x1(0x613b bytes) 2024-09-17T22:57:55.466 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:55.466 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:55.480 INFO:teuthology.orchestra.run.smithi159.stdout:2723 read 0x33f1250 thru 0x33f1edf (0xc90 bytes) 2024-09-17T22:57:55.480 INFO:teuthology.orchestra.run.smithi159.stdout:2724 read 0x16b92f4 thru 0x16ba678 (0x1385 bytes) 2024-09-17T22:57:55.480 INFO:teuthology.orchestra.run.smithi159.stdout:2725 punch from 0xa5a0988 to 0xa5a2e62, (0x24da bytes) 2024-09-17T22:57:55.480 INFO:teuthology.orchestra.run.smithi159.stdout:2726 read 0xbd466c9 thru 0xbd4771b (0x1053 bytes) 2024-09-17T22:57:55.481 INFO:teuthology.orchestra.run.smithi159.stdout:2728 read 0xe56a833 thru 0xe56f32f (0x4afd bytes) 2024-09-17T22:57:55.481 INFO:teuthology.orchestra.run.smithi159.stdout:2729 read 0x45c6118 thru 0x45d061c (0xa505 bytes) 2024-09-17T22:57:55.481 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.486 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:55.486 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:55.494 INFO:teuthology.orchestra.run.smithi159.stdout:2731 trunc from 0xe7cca2e to 0x3e4fb5f 2024-09-17T22:57:55.501 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:55.502 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:2732 read 0x3b4f0d7 thru 0x3b5c1a9 (0xd0d3 bytes) 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:2733 read 0x1c6aae2 thru 0x1c6d951 (0x2e70 bytes) 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:2735 read 0x615c18 thru 0x618485 (0x286e bytes) 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.505 INFO:teuthology.orchestra.run.smithi159.stdout:2739 punch from 0x338e19e to 0x3399fa8, (0xbe0a bytes) 2024-09-17T22:57:55.509 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:55.506+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:55.510 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:55.511 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:55.511 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:55.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:55.516 INFO:teuthology.orchestra.run.smithi159.stdout:2740 trunc from 0x3e4fb5f to 0xccd6e3c 2024-09-17T22:57:55.526 INFO:teuthology.orchestra.run.smithi159.stdout:2741 write 0xcf21392 thru 0xcf246b8 (0x3327 bytes) 2024-09-17T22:57:55.530 INFO:teuthology.orchestra.run.smithi159.stdout:2743 read 0x66ec8b4 thru 0x66ed229 (0x976 bytes) 2024-09-17T22:57:55.530 INFO:teuthology.orchestra.run.smithi159.stdout:2744 write 0x84bd03d thru 0x84cc4db (0xf49f bytes) 2024-09-17T22:57:55.530 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:55.531 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:55.533 INFO:teuthology.orchestra.run.smithi159.stdout:2745 read 0x670827f thru 0x6713967 (0xb6e9 bytes) 2024-09-17T22:57:55.533 INFO:teuthology.orchestra.run.smithi159.stdout:2747 write 0x27ae4a3 thru 0x27bbf17 (0xda75 bytes) 2024-09-17T22:57:55.534 INFO:teuthology.orchestra.run.smithi159.stdout:2749 read 0xcc95e3f thru 0xcc9cb4e (0x6d10 bytes) 2024-09-17T22:57:55.538 INFO:teuthology.orchestra.run.smithi159.stdout:2751 trunc from 0xcf246b9 to 0xa1ac793 2024-09-17T22:57:55.549 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:55.549 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:55.550 INFO:teuthology.orchestra.run.smithi159.stdout:2752 punch from 0xa0b1413 to 0xa0b4a08, (0x35f5 bytes) 2024-09-17T22:57:55.550 INFO:teuthology.orchestra.run.smithi159.stdout:2753 read 0x35e27d4 thru 0x35e4e74 (0x26a1 bytes) 2024-09-17T22:57:55.550 INFO:teuthology.orchestra.run.smithi159.stdout:2754 read 0x8f5d4e3 thru 0x8f64b8b (0x76a9 bytes) 2024-09-17T22:57:55.550 INFO:teuthology.orchestra.run.smithi159.stdout:2755 read 0x6b574ea thru 0x6b60363 (0x8e7a bytes) 2024-09-17T22:57:55.552 INFO:teuthology.orchestra.run.smithi159.stdout:2756 trunc from 0xa1ac793 to 0xc0f3417 2024-09-17T22:57:55.553 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:55.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:55.559 INFO:teuthology.orchestra.run.smithi159.stdout:2757 read 0x375f798 thru 0x37649f6 (0x525f bytes) 2024-09-17T22:57:55.559 INFO:teuthology.orchestra.run.smithi159.stdout:2758 read 0x80d6633 thru 0x80e4fde (0xe9ac bytes) 2024-09-17T22:57:55.559 INFO:teuthology.orchestra.run.smithi159.stdout:2760 punch from 0x67c7193 to 0x67cba29, (0x4896 bytes) 2024-09-17T22:57:55.559 INFO:teuthology.orchestra.run.smithi159.stdout:2761 read 0x712d06c thru 0x713b75d (0xe6f2 bytes) 2024-09-17T22:57:55.559 INFO:teuthology.orchestra.run.smithi159.stdout:2762 write 0xa546606 thru 0xa54a364 (0x3d5f bytes) 2024-09-17T22:57:55.561 INFO:teuthology.orchestra.run.smithi159.stdout:2765 punch from 0xa59c9b8 to 0xa59d625, (0xc6d bytes) 2024-09-17T22:57:55.570 INFO:teuthology.orchestra.run.smithi159.stdout:2767 write 0xeb1959a thru 0xeb1ebcc (0x5633 bytes) 2024-09-17T22:57:55.574 INFO:teuthology.orchestra.run.smithi159.stdout:2768 write 0xc7ece9d thru 0xc7f83ea (0xb54e bytes) 2024-09-17T22:57:55.583 INFO:teuthology.orchestra.run.smithi159.stdout:2770 trunc from 0xeb1ebcd to 0x9c60cf4 2024-09-17T22:57:55.603 INFO:teuthology.orchestra.run.smithi159.stdout:2771 trunc from 0x9c60cf4 to 0x59d4596 2024-09-17T22:57:55.610 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:55.608+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:55.613 INFO:teuthology.orchestra.run.smithi159.stdout:2773 writesame 0x88407 thru 0x93a8a data_size 0x2(0xb684 bytes) 2024-09-17T22:57:55.629 INFO:teuthology.orchestra.run.smithi159.stdout:2774 trunc from 0x59d4596 to 0x96975a6 2024-09-17T22:57:55.631 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:55.631 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:55.634 INFO:teuthology.orchestra.run.smithi159.stdout:2775 write 0x1efeff thru 0x1f3990 (0x3a92 bytes) 2024-09-17T22:57:55.636 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.637 INFO:teuthology.orchestra.run.smithi159.stdout:2777 punch from 0x58c84a2 to 0x58cbbb8, (0x3716 bytes) 2024-09-17T22:57:55.637 INFO:teuthology.orchestra.run.smithi159.stdout:2778 punch from 0x633cca4 to 0x6346713, (0x9a6f bytes) 2024-09-17T22:57:55.637 INFO:teuthology.orchestra.run.smithi159.stdout:2780 read 0x640ef26 thru 0x6417644 (0x871f bytes) 2024-09-17T22:57:55.643 INFO:teuthology.orchestra.run.smithi159.stdout:2781 trunc from 0x96975a6 to 0xe151c80 2024-09-17T22:57:55.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:55.648 INFO:teuthology.orchestra.run.smithi159.stdout:2782 punch from 0x292169e to 0x2921c30, (0x592 bytes) 2024-09-17T22:57:55.648 INFO:teuthology.orchestra.run.smithi159.stdout:2783 read 0x24368c7 thru 0x243f7e1 (0x8f1b bytes) 2024-09-17T22:57:55.648 INFO:teuthology.orchestra.run.smithi159.stdout:2784 read 0x355e6ee thru 0x356ade5 (0xc6f8 bytes) 2024-09-17T22:57:55.649 INFO:teuthology.orchestra.run.smithi159.stdout:2787 write 0x22b61cc thru 0x22bb890 (0x56c5 bytes) 2024-09-17T22:57:55.651 INFO:teuthology.orchestra.run.smithi159.stdout:2788 read 0x5832b30 thru 0x583a88e (0x7d5f bytes) 2024-09-17T22:57:55.655 INFO:teuthology.orchestra.run.smithi159.stdout:2790 trunc from 0xe151c80 to 0xa0af4c0 2024-09-17T22:57:55.660 INFO:teuthology.orchestra.run.smithi159.stdout:2791 punch from 0x837e1bb to 0x8385a98, (0x78dd bytes) 2024-09-17T22:57:55.662 INFO:teuthology.orchestra.run.smithi159.stdout:2792 trunc from 0xa0af4c0 to 0xb2c064d 2024-09-17T22:57:55.676 INFO:teuthology.orchestra.run.smithi159.stdout:2794 trunc from 0xb2c064d to 0x3e2e52f 2024-09-17T22:57:55.688 INFO:teuthology.orchestra.run.smithi159.stdout:2796 writesame 0x4b6a336 thru 0x4b6cf90 data_size 0x5(0x2c5b bytes) 2024-09-17T22:57:55.704 INFO:teuthology.orchestra.run.smithi159.stdout:2799 write 0xcc16014 thru 0xcc22fb8 (0xcfa5 bytes) 2024-09-17T22:57:55.711 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:55.708+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:55.711 INFO:teuthology.orchestra.run.smithi159.stdout:2801 trunc from 0xcc22fb9 to 0x9fdad34 2024-09-17T22:57:55.727 INFO:teuthology.orchestra.run.smithi159.stdout:2803 read 0x10873ef thru 0x108eec4 (0x7ad6 bytes) 2024-09-17T22:57:55.727 INFO:teuthology.orchestra.run.smithi159.stdout:2804 read 0x3027701 thru 0x302c184 (0x4a84 bytes) 2024-09-17T22:57:55.727 INFO:teuthology.orchestra.run.smithi159.stdout:2805 trunc from 0x9fdad34 to 0xa6a21d7 2024-09-17T22:57:55.733 INFO:teuthology.orchestra.run.smithi159.stdout:2807 read 0x7f88ab8 thru 0x7f954f8 (0xca41 bytes) 2024-09-17T22:57:55.733 INFO:teuthology.orchestra.run.smithi159.stdout:2810 write 0x5c56616 thru 0x5c61478 (0xae63 bytes) 2024-09-17T22:57:55.737 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:55.737 INFO:teuthology.orchestra.run.smithi159.stdout:2812 clone 23 order 21 su 65536 sc 15 2024-09-17T22:57:55.754 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:55.754 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:55.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:55.811 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:55.809+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:55.866 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:55.866 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:55.876 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:55.916 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:55.914+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:56.013 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:56.011+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:56.018 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.019 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.027 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:56.114 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:56.112+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:56.131 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.132 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.142 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:56.215 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:56.212+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:56.224 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone22 from 0xa6a21d7 (overlap 0x1eeb8b) to 0x15a4e1 2024-09-17T22:57:56.293 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.293 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.302 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:56.312 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T22:57:56.313 INFO:teuthology.orchestra.run.smithi159.stdout:2813 write 0x51a10c8 thru 0x51a5428 (0x4361 bytes) 2024-09-17T22:57:56.315 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:56.313+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:56.320 INFO:teuthology.orchestra.run.smithi159.stdout:2816 writesame 0x6a0a301 thru 0x6a17ace data_size 0x2(0xd7ce bytes) 2024-09-17T22:57:56.332 INFO:teuthology.orchestra.run.smithi159.stdout:2817 trunc from 0xa6a21d7 to 0xa8e7a41 2024-09-17T22:57:56.338 INFO:teuthology.orchestra.run.smithi159.stdout:2818 read 0x3e0860e thru 0x3e0d0e4 (0x4ad7 bytes) 2024-09-17T22:57:56.338 INFO:teuthology.orchestra.run.smithi159.stdout:2820 write 0x641ba7e thru 0x6428ae9 (0xd06c bytes) 2024-09-17T22:57:56.344 INFO:teuthology.orchestra.run.smithi159.stdout:2822 write 0x84587e1 thru 0x8464d8a (0xc5aa bytes) 2024-09-17T22:57:56.348 INFO:teuthology.orchestra.run.smithi159.stdout:2824 read 0x20df479 thru 0x20e9c3c (0xa7c4 bytes) 2024-09-17T22:57:56.348 INFO:teuthology.orchestra.run.smithi159.stdout:2825 writesame 0x52d328f thru 0x52e085e data_size 0x58(0xd5d0 bytes) 2024-09-17T22:57:56.352 INFO:teuthology.orchestra.run.smithi159.stdout:2826 write 0x29959b7 thru 0x29a36b5 (0xdcff bytes) 2024-09-17T22:57:56.357 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.367 INFO:teuthology.orchestra.run.smithi159.stdout:2829 write 0xe5fdca6 thru 0xe607fb8 (0xa313 bytes) 2024-09-17T22:57:56.371 INFO:teuthology.orchestra.run.smithi159.stdout:2830 read 0x281d84a thru 0x2820b98 (0x334f bytes) 2024-09-17T22:57:56.371 INFO:teuthology.orchestra.run.smithi159.stdout:2831 punch from 0x99636e3 to 0x99722f0, (0xec0d bytes) 2024-09-17T22:57:56.371 INFO:teuthology.orchestra.run.smithi159.stdout:2832 read 0xe18e3a7 thru 0xe197a3e (0x9698 bytes) 2024-09-17T22:57:56.372 INFO:teuthology.orchestra.run.smithi159.stdout:2834 write 0xd6be376 thru 0xd6c23b3 (0x403e bytes) 2024-09-17T22:57:56.377 INFO:teuthology.orchestra.run.smithi159.stdout:2835 trunc from 0xe607fb9 to 0xc54fcb1 2024-09-17T22:57:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:2836 write 0x314fb7c thru 0x315b353 (0xb7d8 bytes) 2024-09-17T22:57:56.394 INFO:teuthology.orchestra.run.smithi159.stdout:2838 write 0x4bf12f0 thru 0x4c003f4 (0xf105 bytes) 2024-09-17T22:57:56.402 INFO:teuthology.orchestra.run.smithi159.stdout:2839 trunc from 0xc54fcb1 to 0xe61ccbf 2024-09-17T22:57:56.410 INFO:teuthology.orchestra.run.smithi159.stdout:2840 read 0x78436eb thru 0x784e76d (0xb083 bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2841 punch from 0x40195ee to 0x401efe9, (0x59fb bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2843 punch from 0x840cf3c to 0x8415743, (0x8807 bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2844 read 0x4a21ec6 thru 0x4a30a02 (0xeb3d bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2846 read 0x1c62d3b thru 0x1c72a35 (0xfcfb bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2847 punch from 0xe1ced36 to 0xe1ddf6c, (0xf236 bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2848 read 0x5f41c05 thru 0x5f494e3 (0x78df bytes) 2024-09-17T22:57:56.411 INFO:teuthology.orchestra.run.smithi159.stdout:2849 write 0x1fee10a thru 0x1ff0c1e (0x2b15 bytes) 2024-09-17T22:57:56.414 INFO:teuthology.orchestra.run.smithi159.stdout:2850 read 0xaa52d97 thru 0xaa583ff (0x5669 bytes) 2024-09-17T22:57:56.415 INFO:teuthology.orchestra.run.smithi159.stdout:2851 write 0xc60c52c thru 0xc6148ae (0x8383 bytes) 2024-09-17T22:57:56.415 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:56.413+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:56.417 INFO:teuthology.orchestra.run.smithi159.stdout:2854 write 0x24214dc thru 0x242ed4f (0xd874 bytes) 2024-09-17T22:57:56.420 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.421 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.421 INFO:teuthology.orchestra.run.smithi159.stdout:2856 punch from 0xab7e095 to 0xab8061a, (0x2585 bytes) 2024-09-17T22:57:56.423 INFO:teuthology.orchestra.run.smithi159.stdout:2857 trunc from 0xe61ccbf to 0xafd4fc7 2024-09-17T22:57:56.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:56.448 INFO:teuthology.orchestra.run.smithi159.stdout:2858 read 0x66761bc thru 0x668048f (0xa2d4 bytes) 2024-09-17T22:57:56.448 INFO:teuthology.orchestra.run.smithi159.stdout:2859 writesame 0x5890b02 thru 0x589db6c data_size 0x5(0xd06b bytes) 2024-09-17T22:57:56.455 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.455 INFO:teuthology.orchestra.run.smithi159.stdout:2861 write 0xa19ad49 thru 0xa19f3be (0x4676 bytes) 2024-09-17T22:57:56.456 INFO:teuthology.orchestra.run.smithi159.stdout:2863 flatten 2024-09-17T22:57:56.483 INFO:teuthology.orchestra.run.smithi159.stdout:2864 read 0x57bcbb8 thru 0x57c4e8e (0x82d7 bytes) 2024-09-17T22:57:56.483 INFO:teuthology.orchestra.run.smithi159.stdout:2867 write 0x49604d8 thru 0x496a2a6 (0x9dcf bytes) 2024-09-17T22:57:56.486 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.486 INFO:teuthology.orchestra.run.smithi159.stdout:2870 write 0x50306fd thru 0x50308d5 (0x1d9 bytes) 2024-09-17T22:57:56.488 INFO:teuthology.orchestra.run.smithi159.stdout:2871 read 0xa6513e7 thru 0xa65af59 (0x9b73 bytes) 2024-09-17T22:57:56.489 INFO:teuthology.orchestra.run.smithi159.stdout:2872 read 0x469f291 thru 0x46a68a2 (0x7612 bytes) 2024-09-17T22:57:56.499 INFO:teuthology.orchestra.run.smithi159.stdout:2873 trunc from 0xafd4fc7 to 0x32315a8 2024-09-17T22:57:56.517 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:56.515+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:56.526 INFO:teuthology.orchestra.run.smithi159.stdout:2877 writesame 0x7435eff thru 0x7436ab9 data_size 0x21(0xbbb bytes) 2024-09-17T22:57:56.537 INFO:teuthology.orchestra.run.smithi159.stdout:2878 write 0xba29661 thru 0xba36e31 (0xd7d1 bytes) 2024-09-17T22:57:56.543 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.550 INFO:teuthology.orchestra.run.smithi159.stdout:2880 write 0xd661732 thru 0xd66ca78 (0xb347 bytes) 2024-09-17T22:57:56.553 INFO:teuthology.orchestra.run.smithi159.stdout:2882 write 0x313a070 thru 0x313ab58 (0xae9 bytes) 2024-09-17T22:57:56.569 INFO:teuthology.orchestra.run.smithi159.stdout:2883 trunc from 0xd66ca79 to 0x28b1c8b 2024-09-17T22:57:56.581 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.581 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.591 INFO:teuthology.orchestra.run.smithi159.stdout:2884 write 0x508ce4a thru 0x508d9f4 (0xbab bytes) 2024-09-17T22:57:56.591 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:56.596 INFO:teuthology.orchestra.run.smithi159.stdout:2886 trunc from 0x508d9f5 to 0x169ebca 2024-09-17T22:57:56.618 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:56.615+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:56.625 INFO:teuthology.orchestra.run.smithi159.stdout:2887 write 0x942719f thru 0x943165d (0xa4bf bytes) 2024-09-17T22:57:56.629 INFO:teuthology.orchestra.run.smithi159.stdout:2888 read 0x156b7c0 thru 0x157b34e (0xfb8f bytes) 2024-09-17T22:57:56.629 INFO:teuthology.orchestra.run.smithi159.stdout:2889 punch from 0x31b5476 to 0x31b82ff, (0x2e89 bytes) 2024-09-17T22:57:56.642 INFO:teuthology.orchestra.run.smithi159.stdout:2891 write 0xead21bf thru 0xead642a (0x426c bytes) 2024-09-17T22:57:56.644 INFO:teuthology.orchestra.run.smithi159.stdout:2892 writesame 0xb81a8c6 thru 0xb82225f data_size 0x16(0x799a bytes) 2024-09-17T22:57:56.648 INFO:teuthology.orchestra.run.smithi159.stdout:2893 read 0x36a016e thru 0x36b0020 (0xfeb3 bytes) 2024-09-17T22:57:56.648 INFO:teuthology.orchestra.run.smithi159.stdout:2894 write 0xdf6fc4 thru 0xdf7a57 (0xa94 bytes) 2024-09-17T22:57:56.651 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.651 INFO:teuthology.orchestra.run.smithi159.stdout:2896 read 0xa6aa978 thru 0xa6b4e87 (0xa510 bytes) 2024-09-17T22:57:56.651 INFO:teuthology.orchestra.run.smithi159.stdout:2898 read 0xcc56cf0 thru 0xcc5b288 (0x4599 bytes) 2024-09-17T22:57:56.651 INFO:teuthology.orchestra.run.smithi159.stdout:2899 read 0x1099bc8 thru 0x10a364a (0x9a83 bytes) 2024-09-17T22:57:56.652 INFO:teuthology.orchestra.run.smithi159.stdout:2900 punch from 0xb664f43 to 0xb66cac4, (0x7b81 bytes) 2024-09-17T22:57:56.652 INFO:teuthology.orchestra.run.smithi159.stdout:2901 punch from 0xbb375ef to 0xbb380eb, (0xafc bytes) 2024-09-17T22:57:56.656 INFO:teuthology.orchestra.run.smithi159.stdout:2902 trunc from 0xead642b to 0xb31ba2f 2024-09-17T22:57:56.666 INFO:teuthology.orchestra.run.smithi159.stdout:2903 read 0x234c2e1 thru 0x235a505 (0xe225 bytes) 2024-09-17T22:57:56.667 INFO:teuthology.orchestra.run.smithi159.stdout:2906 write 0x5ca9348 thru 0x5cb588e (0xc547 bytes) 2024-09-17T22:57:56.670 INFO:teuthology.orchestra.run.smithi159.stdout:2907 write 0x35e8790 thru 0x35ee073 (0x58e4 bytes) 2024-09-17T22:57:56.673 INFO:teuthology.orchestra.run.smithi159.stdout:2908 write 0x2ec394 thru 0x2ec77b (0x3e8 bytes) 2024-09-17T22:57:56.674 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.674 INFO:teuthology.orchestra.run.smithi159.stdout:2910 read 0x9a29ead thru 0x9a336a7 (0x97fb bytes) 2024-09-17T22:57:56.681 INFO:teuthology.orchestra.run.smithi159.stdout:2911 write 0xc99b5ce thru 0xc9a7593 (0xbfc6 bytes) 2024-09-17T22:57:56.700 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.700 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.702 INFO:teuthology.orchestra.run.smithi159.stdout:2913 trunc from 0xc9a7594 to 0x829b2c 2024-09-17T22:57:56.710 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:57:56.722 INFO:teuthology.orchestra.run.smithi159.stdout:2914 trunc from 0x829b2c to 0x75bb6a6 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:2917 read 0x532c796 thru 0x5338798 (0xc003 bytes) 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:2919 read 0x48f535e thru 0x49020d3 (0xcd76 bytes) 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:2921 read 0x5b64b9a thru 0x5b6f113 (0xa57a bytes) 2024-09-17T22:57:56.729 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.744 INFO:teuthology.orchestra.run.smithi159.stdout:2923 write 0xe716256 thru 0xe71e37d (0x8128 bytes) 2024-09-17T22:57:56.748 INFO:teuthology.orchestra.run.smithi159.stdout:2924 read 0x270bc3f thru 0x2718c85 (0xd047 bytes) 2024-09-17T22:57:56.748 INFO:teuthology.orchestra.run.smithi159.stdout:2925 read 0x4f613a3 thru 0x4f69e0b (0x8a69 bytes) 2024-09-17T22:57:56.748 INFO:teuthology.orchestra.run.smithi159.stdout:2926 writesame 0x7f72dfa thru 0x7f7f2c3 data_size 0x1(0xc4ca bytes) 2024-09-17T22:57:56.754 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:56.752+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:56.775 INFO:teuthology.orchestra.run.smithi159.stdout:2928 trunc from 0xe71e37e to 0xea35542 2024-09-17T22:57:56.780 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:56.781 INFO:teuthology.orchestra.run.smithi159.stdout:2930 writesame 0x2a72b8e thru 0x2a7734e data_size 0x27(0x47c1 bytes) 2024-09-17T22:57:56.783 INFO:teuthology.orchestra.run.smithi159.stdout:2932 clone 24 order 23 su 32768 sc 11 2024-09-17T22:57:56.817 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.817 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.820 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:56.818+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:56.825 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:57:56.921 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:56.919+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:56.928 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:56.929 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:56.937 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:57:57.022 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:57.020+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:57.040 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.040 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:57:57.123 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:57.120+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:57.155 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.155 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.165 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:57:57.209 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:57:57.224 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:57.221+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:57.316 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.316 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.325 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:57.323+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:57.326 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:57:57.377 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T22:57:57.384 INFO:teuthology.orchestra.run.smithi159.stdout:2934 write 0xe873f92 thru 0xe876f14 (0x2f83 bytes) 2024-09-17T22:57:57.393 INFO:teuthology.orchestra.run.smithi159.stdout:2935 punch from 0x846e2c to 0x84feaf, (0x9083 bytes) 2024-09-17T22:57:57.393 INFO:teuthology.orchestra.run.smithi159.stdout:2936 writesame 0x8c0c7b3 thru 0x8c19735 data_size 0x1(0xcf83 bytes) 2024-09-17T22:57:57.420 INFO:teuthology.orchestra.run.smithi159.stdout:2937 write 0xbc29699 thru 0xbc29902 (0x26a bytes) 2024-09-17T22:57:57.426 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:57.423+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:57.442 INFO:teuthology.orchestra.run.smithi159.stdout:2938 trunc from 0xea35542 to 0x17e2a67 2024-09-17T22:57:57.443 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.444 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.457 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:57:57.498 INFO:teuthology.orchestra.run.smithi159.stdout:2940 read 0x14c287f thru 0x14c8a6e (0x61f0 bytes) 2024-09-17T22:57:57.498 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.499 INFO:teuthology.orchestra.run.smithi159.stdout:2942 read 0x122bfd8 thru 0x123781b (0xb844 bytes) 2024-09-17T22:57:57.510 INFO:teuthology.orchestra.run.smithi159.stdout:2943 write 0x501e880 thru 0x502c059 (0xd7da bytes) 2024-09-17T22:57:57.524 INFO:teuthology.orchestra.run.smithi159.stdout:2944 write 0xaf6fbdb thru 0xaf7ce55 (0xd27b bytes) 2024-09-17T22:57:57.526 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:57.524+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:57.529 INFO:teuthology.orchestra.run.smithi159.stdout:2945 read 0x6300017 thru 0x6301425 (0x140f bytes) 2024-09-17T22:57:57.529 INFO:teuthology.orchestra.run.smithi159.stdout:2946 writesame 0x6c7988c thru 0x6c86e9c data_size 0x3(0xd611 bytes) 2024-09-17T22:57:57.537 INFO:teuthology.orchestra.run.smithi159.stdout:2948 punch from 0x33e9d3e to 0x33f757a, (0xd83c bytes) 2024-09-17T22:57:57.537 INFO:teuthology.orchestra.run.smithi159.stdout:2949 write 0x25a055b thru 0x25a8c87 (0x872d bytes) 2024-09-17T22:57:57.539 INFO:teuthology.orchestra.run.smithi159.stdout:2950 read 0x46855fe thru 0x46875ba (0x1fbd bytes) 2024-09-17T22:57:57.539 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.551 INFO:teuthology.orchestra.run.smithi159.stdout:2952 trunc from 0xaf7ce56 to 0xcc7fbc 2024-09-17T22:57:57.555 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:57:57.566 INFO:teuthology.orchestra.run.smithi159.stdout:2953 read 0xe5aea thru 0xecb97 (0x70ae bytes) 2024-09-17T22:57:57.567 INFO:teuthology.orchestra.run.smithi159.stdout:2955 read 0xa6c468 thru 0xa7a544 (0xe0dd bytes) 2024-09-17T22:57:57.587 INFO:teuthology.orchestra.run.smithi159.stdout:2957 writesame 0xc4dbe63 thru 0xc4e1afe data_size 0x2(0x5c9c bytes) 2024-09-17T22:57:57.587 INFO:teuthology.orchestra.run.smithi159.stdout:2958 writesame 0x20485ee thru 0x20567e7 data_size 0x82(0xe1fa bytes) 2024-09-17T22:57:57.589 INFO:teuthology.orchestra.run.smithi159.stdout:2959 write 0x9b805ad thru 0x9b8bd8d (0xb7e1 bytes) 2024-09-17T22:57:57.593 INFO:teuthology.orchestra.run.smithi159.stdout:2963 punch from 0x7b4ad64 to 0x7b4ec22, (0x3ebe bytes) 2024-09-17T22:57:57.593 INFO:teuthology.orchestra.run.smithi159.stdout:2964 punch from 0x9091d14 to 0x9093aaf, (0x1d9b bytes) 2024-09-17T22:57:57.593 INFO:teuthology.orchestra.run.smithi159.stdout:2965 read 0x35baab1 thru 0x35c2bd9 (0x8129 bytes) 2024-09-17T22:57:57.593 INFO:teuthology.orchestra.run.smithi159.stdout:2966 read 0x5f688b8 thru 0x5f68b8a (0x2d3 bytes) 2024-09-17T22:57:57.594 INFO:teuthology.orchestra.run.smithi159.stdout:2967 write 0x4dfe976 thru 0x4e047a8 (0x5e33 bytes) 2024-09-17T22:57:57.595 INFO:teuthology.orchestra.run.smithi159.stdout:2968 write 0xad25364 thru 0xad25b32 (0x7cf bytes) 2024-09-17T22:57:57.598 INFO:teuthology.orchestra.run.smithi159.stdout:2972 write 0x80f4e29 thru 0x8102789 (0xd961 bytes) 2024-09-17T22:57:57.603 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.603 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.605 INFO:teuthology.orchestra.run.smithi159.stdout:2973 read 0x5d30dfe thru 0x5d40b94 (0xfd97 bytes) 2024-09-17T22:57:57.606 INFO:teuthology.orchestra.run.smithi159.stdout:2976 write 0x5c0cca thru 0x5c6bdb (0x5f12 bytes) 2024-09-17T22:57:57.616 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:57:57.617 INFO:teuthology.orchestra.run.smithi159.stdout:2978 writesame 0xdba425a thru 0xdbb256f data_size 0x3(0xe316 bytes) 2024-09-17T22:57:57.617 INFO:teuthology.orchestra.run.smithi159.stdout:2979 write 0x14579cf thru 0x14662a0 (0xe8d2 bytes) 2024-09-17T22:57:57.617 INFO:teuthology.orchestra.run.smithi159.stdout:2980 write 0x8447260 thru 0x845303f (0xbde0 bytes) 2024-09-17T22:57:57.622 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.622 INFO:teuthology.orchestra.run.smithi159.stdout:2982 write 0x622091a thru 0x622cd09 (0xc3f0 bytes) 2024-09-17T22:57:57.625 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.625 INFO:teuthology.orchestra.run.smithi159.stdout:2985 read 0xb7b5827 thru 0xb7bce05 (0x75df bytes) 2024-09-17T22:57:57.625 INFO:teuthology.orchestra.run.smithi159.stdout:2986 trunc from 0xdbb2570 to 0xddaca37 2024-09-17T22:57:57.626 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:57.624+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:57.631 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.631 INFO:teuthology.orchestra.run.smithi159.stdout:2988 punch from 0x7a15dbb to 0x7a2025f, (0xa4a4 bytes) 2024-09-17T22:57:57.631 INFO:teuthology.orchestra.run.smithi159.stdout:2989 write 0xa1f6f14 thru 0xa1fd127 (0x6214 bytes) 2024-09-17T22:57:57.633 INFO:teuthology.orchestra.run.smithi159.stdout:2990 trunc from 0xddaca37 to 0xd58fcaa 2024-09-17T22:57:57.639 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.639 INFO:teuthology.orchestra.run.smithi159.stdout:2992 write 0xb30ef96 thru 0xb316df4 (0x7e5f bytes) 2024-09-17T22:57:57.643 INFO:teuthology.orchestra.run.smithi159.stdout:2993 write 0x6dd132a thru 0x6dda667 (0x933e bytes) 2024-09-17T22:57:57.644 INFO:teuthology.orchestra.run.smithi159.stdout:2994 write 0x68b71f3 thru 0x68b7f95 (0xda3 bytes) 2024-09-17T22:57:57.651 INFO:teuthology.orchestra.run.smithi159.stdout:2995 trunc from 0xd58fcaa to 0x90c4b20 2024-09-17T22:57:57.664 INFO:teuthology.orchestra.run.smithi159.stdout:2996 read 0x8dde2f4 thru 0x8de4b1e (0x682b bytes) 2024-09-17T22:57:57.664 INFO:teuthology.orchestra.run.smithi159.stdout:2998 write 0x3fbcec2 thru 0x3fc5ff4 (0x9133 bytes) 2024-09-17T22:57:57.666 INFO:teuthology.orchestra.run.smithi159.stdout:3000 punch from 0x56e72c3 to 0x56ee29a, (0x6fd7 bytes) 2024-09-17T22:57:57.666 INFO:teuthology.orchestra.run.smithi159.stdout:3001 writesame 0x50d6f9f thru 0x50e4f35 data_size 0x25(0xdf97 bytes) 2024-09-17T22:57:57.678 INFO:teuthology.orchestra.run.smithi159.stdout:3002 write 0xc2e4ef1 thru 0xc2ef7fb (0xa90b bytes) 2024-09-17T22:57:57.685 INFO:teuthology.orchestra.run.smithi159.stdout:3005 write 0x9e6fc71 thru 0x9e7c5ba (0xc94a bytes) 2024-09-17T22:57:57.690 INFO:teuthology.orchestra.run.smithi159.stdout:3007 write 0x5c2b24 thru 0x5c44e5 (0x19c2 bytes) 2024-09-17T22:57:57.691 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.698 INFO:teuthology.orchestra.run.smithi159.stdout:3010 write 0xcbf45bc thru 0xcbfa804 (0x6249 bytes) 2024-09-17T22:57:57.701 INFO:teuthology.orchestra.run.smithi159.stdout:3011 read 0xa746291 thru 0xa749038 (0x2da8 bytes) 2024-09-17T22:57:57.702 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.702 INFO:teuthology.orchestra.run.smithi159.stdout:3015 write 0x3d89dbc thru 0x3d958f9 (0xbb3e bytes) 2024-09-17T22:57:57.703 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.703 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.703 INFO:teuthology.orchestra.run.smithi159.stdout:3019 punch from 0x9bc56ce to 0x9bd387e, (0xe1b0 bytes) 2024-09-17T22:57:57.710 INFO:teuthology.orchestra.run.smithi159.stdout:3020 writesame 0xda54d69 thru 0xda5a72c data_size 0x1e(0x59c4 bytes) 2024-09-17T22:57:57.710 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:57.711 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:57.713 INFO:teuthology.orchestra.run.smithi159.stdout:3022 trunc from 0xda5a72d to 0xb361b1a 2024-09-17T22:57:57.720 INFO:teuthology.orchestra.run.smithi159.stdout:3023 writesame 0x92e8b68 thru 0x92eaa40 data_size 0x1(0x1ed9 bytes) 2024-09-17T22:57:57.723 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.723 INFO:teuthology.orchestra.run.smithi159.stdout:3027 writesame 0x9c43705 thru 0x9c46b9f data_size 0x3(0x349b bytes) 2024-09-17T22:57:57.726 INFO:teuthology.orchestra.run.smithi159.stdout:3028 read 0x42f6ed3 thru 0x430648d (0xf5bb bytes) 2024-09-17T22:57:57.727 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:57.724+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:57.727 INFO:teuthology.orchestra.run.smithi159.stdout:3029 read 0x2a37c03 thru 0x2a4185b (0x9c59 bytes) 2024-09-17T22:57:57.727 INFO:teuthology.orchestra.run.smithi159.stdout:3030 write 0x2cc00ee thru 0x2cc5336 (0x5249 bytes) 2024-09-17T22:57:57.728 INFO:teuthology.orchestra.run.smithi159.stdout:3033 write 0xa948bdf thru 0xa952611 (0x9a33 bytes) 2024-09-17T22:57:57.730 INFO:teuthology.orchestra.run.smithi159.stdout:3034 punch from 0xa86fdd6 to 0xa870501, (0x72b bytes) 2024-09-17T22:57:57.730 INFO:teuthology.orchestra.run.smithi159.stdout:3035 punch from 0x243c9c3 to 0x243e83f, (0x1e7c bytes) 2024-09-17T22:57:57.730 INFO:teuthology.orchestra.run.smithi159.stdout:3036 punch from 0xaa52c03 to 0xaa60449, (0xd846 bytes) 2024-09-17T22:57:57.730 INFO:teuthology.orchestra.run.smithi159.stdout:3037 read 0xb02d9e thru 0xb0f258 (0xc4bb bytes) 2024-09-17T22:57:57.731 INFO:teuthology.orchestra.run.smithi159.stdout:3038 trunc from 0xb361b1a to 0xb821a02 2024-09-17T22:57:57.733 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:57.733 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:57.746 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:57:57.746 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:57:57.748 INFO:teuthology.orchestra.run.smithi159.stdout:3039 trunc from 0xb821a02 to 0x14dbf0a 2024-09-17T22:57:57.751 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:57:57.751 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:57:57.761 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:57:57.772 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:57:57.773 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:57:57.777 INFO:teuthology.orchestra.run.smithi159.stdout:3040 write 0x6f44e5c thru 0x6f4fa91 (0xac36 bytes) 2024-09-17T22:57:57.781 INFO:teuthology.orchestra.run.smithi159.stdout:3043 write 0x37f55bf thru 0x37fceb3 (0x78f5 bytes) 2024-09-17T22:57:57.783 INFO:teuthology.orchestra.run.smithi159.stdout:3045 read 0x3d86045 thru 0x3d92053 (0xc00f bytes) 2024-09-17T22:57:57.783 INFO:teuthology.orchestra.run.smithi159.stdout:3046 punch from 0xa7361c to 0xa7f1ec, (0xbbd0 bytes) 2024-09-17T22:57:57.790 INFO:teuthology.orchestra.run.smithi159.stdout:3047 trunc from 0x6f4fa92 to 0xaa7c8f 2024-09-17T22:57:57.793 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:57:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:57:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:57:57.814 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:57:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:57:57.815 INFO:teuthology.orchestra.run.smithi159.stdout:3049 write 0x346148d thru 0x346e211 (0xcd85 bytes) 2024-09-17T22:57:57.816 INFO:teuthology.orchestra.run.smithi159.stdout:3050 write 0x17452ab thru 0x1745371 (0xc7 bytes) 2024-09-17T22:57:57.829 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:57.826+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:57.830 INFO:teuthology.orchestra.run.smithi159.stdout:3051 write 0x9467611 thru 0x946a7ce (0x31be bytes) 2024-09-17T22:57:57.834 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:57:57.835 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:57:57.839 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:57:57.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:57:57.843 INFO:teuthology.orchestra.run.smithi159.stdout:3052 write 0xd6fa4c0 thru 0xd6fd10c (0x2c4d bytes) 2024-09-17T22:57:57.853 INFO:teuthology.orchestra.run.smithi159.stdout:3053 write 0xebd3d72 thru 0xebda360 (0x65ef bytes) 2024-09-17T22:57:57.856 INFO:teuthology.orchestra.run.smithi159.stdout:3054 write 0x143afad thru 0x143b3ed (0x441 bytes) 2024-09-17T22:57:57.857 INFO:teuthology.orchestra.run.smithi159.stdout:3055 write 0x1bd349e thru 0x1bd8cd0 (0x5833 bytes) 2024-09-17T22:57:57.859 INFO:teuthology.orchestra.run.smithi159.stdout:3059 read 0x49e7b74 thru 0x49ec8ca (0x4d57 bytes) 2024-09-17T22:57:57.859 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:57.859 INFO:teuthology.orchestra.run.smithi159.stdout:3061 read 0x200183c thru 0x2005fed (0x47b2 bytes) 2024-09-17T22:57:57.859 INFO:teuthology.orchestra.run.smithi159.stdout:3062 writesame 0x5d923fb thru 0x5d9f50b data_size 0x17(0xd111 bytes) 2024-09-17T22:57:57.864 INFO:teuthology.orchestra.run.smithi159.stdout:3063 writesame 0x6af7fbd thru 0x6b03e13 data_size 0x1(0xbe57 bytes) 2024-09-17T22:57:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:3065 read 0x5f5f55b thru 0x5f6c0a9 (0xcb4f bytes) 2024-09-17T22:57:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:3067 write 0xdc2fe7c thru 0xdc35a5b (0x5be0 bytes) 2024-09-17T22:57:57.886 INFO:teuthology.orchestra.run.smithi159.stdout:3068 read 0x8dd1ffe thru 0x8dd47aa (0x27ad bytes) 2024-09-17T22:57:57.886 INFO:teuthology.orchestra.run.smithi159.stdout:3069 write 0x4649bd4 thru 0x464bacf (0x1efc bytes) 2024-09-17T22:57:57.887 INFO:teuthology.orchestra.run.smithi159.stdout:3071 read 0xc722fa thru 0xc81ddd (0xfae4 bytes) 2024-09-17T22:57:57.902 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:57.903 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:57.903 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:57.903 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:57.903 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:57.903 INFO:teuthology.orchestra.run.smithi159.stdout:3075 trunc from 0xebda361 to 0x178b343 2024-09-17T22:57:57.915 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:57:57.934 INFO:teuthology.orchestra.run.smithi159.stdout:3077 write 0x75f40d7 thru 0x7603e04 (0xfd2e bytes) 2024-09-17T22:57:57.934 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:57:57.931+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:57:57.952 INFO:teuthology.orchestra.run.smithi159.stdout:3078 write 0xa6f719f thru 0xa6fc2b8 (0x511a bytes) 2024-09-17T22:57:57.970 INFO:teuthology.orchestra.run.smithi159.stdout:3079 write 0xdd0f77c thru 0xdd1ba92 (0xc317 bytes) 2024-09-17T22:57:57.976 INFO:teuthology.orchestra.run.smithi159.stdout:3080 write 0x9565c33 thru 0x9573c86 (0xe054 bytes) 2024-09-17T22:57:57.979 INFO:teuthology.orchestra.run.smithi159.stdout:3082 read 0xb2972b2 thru 0xb29e47d (0x71cc bytes) 2024-09-17T22:57:57.979 INFO:teuthology.orchestra.run.smithi159.stdout:3085 read 0x24c5fd5 thru 0x24d3328 (0xd354 bytes) 2024-09-17T22:57:57.979 INFO:teuthology.orchestra.run.smithi159.stdout:3086 read 0xd5e1e1f thru 0xd5e2021 (0x203 bytes) 2024-09-17T22:57:57.980 INFO:teuthology.orchestra.run.smithi159.stdout:3087 write 0x872ceb1 thru 0x873c065 (0xf1b5 bytes) 2024-09-17T22:57:57.991 INFO:teuthology.orchestra.run.smithi159.stdout:3088 trunc from 0xdd1ba93 to 0x70250b3 2024-09-17T22:57:58.003 INFO:teuthology.orchestra.run.smithi159.stdout:3089 writesame 0x584988e thru 0x5854103 data_size 0x1(0xa876 bytes) 2024-09-17T22:57:58.022 INFO:teuthology.orchestra.run.smithi159.stdout:3091 trunc from 0x70250b3 to 0x9a0e5f3 2024-09-17T22:57:58.027 INFO:teuthology.orchestra.run.smithi159.stdout:3093 read 0xcf9b3a thru 0xd0529a (0xb761 bytes) 2024-09-17T22:57:58.028 INFO:teuthology.orchestra.run.smithi159.stdout:3094 read 0x75f359a thru 0x75fb241 (0x7ca8 bytes) 2024-09-17T22:57:58.028 INFO:teuthology.orchestra.run.smithi159.stdout:3095 write 0x5b0c8f3 thru 0x5b1a270 (0xd97e bytes) 2024-09-17T22:57:58.030 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:58.028+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:58.031 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.032 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.032 INFO:teuthology.orchestra.run.smithi159.stdout:3101 punch from 0x1e9685e to 0x1e98d35, (0x24d7 bytes) 2024-09-17T22:57:58.032 INFO:teuthology.orchestra.run.smithi159.stdout:3102 read 0x526f4b6 thru 0x527a8e4 (0xb42f bytes) 2024-09-17T22:57:58.032 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.032 INFO:teuthology.orchestra.run.smithi159.stdout:3105 read 0x5ff609d thru 0x5ffdd3c (0x7ca0 bytes) 2024-09-17T22:57:58.038 INFO:teuthology.orchestra.run.smithi159.stdout:3106 trunc from 0x9a0e5f3 to 0x55d1ab2 2024-09-17T22:57:58.052 INFO:teuthology.orchestra.run.smithi159.stdout:3107 read 0x4245673 thru 0x424b8c1 (0x624f bytes) 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.166874843999999, 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:57:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:57:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1044 3.9 e28/22 3:954c0790:::rbd_data.118840f89003.0000000000000090:head v 28'4, mlcod=25'2)", 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.029073140000001, 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032794199999999999, 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.09e-07 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028800+0000", 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7649999999999999e-05 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028812+0000", 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1865000000000001e-05 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028819+0000", 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7750000000000004e-06 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028870+0000", 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0949000000000003e-05 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028889+0000", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9199999999999999e-05 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032019+0000", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031302230000000001 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032061+0000", 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1949000000000001e-05 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 41 entries 28'5 (0'0) error 3:9656af7d:::rbd_data.118840f89003.0000000000000092:head by client.4488.0:1048 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.028999992999999, 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034247520000000001, 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028863+0000", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7249999999999997e-06 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028879+0000", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5862000000000002e-05 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028892+0000", 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3076e-05 2024-09-17T22:57:58.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.029115+0000", 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000223427 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032279+0000", 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.003164189 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 50 entries 28'7 (0'0) error 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head by client.4488.0:1110 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.018397866999997, 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0077516210000000002, 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039458+0000", 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.99e-07 2024-09-17T22:57:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039466+0000", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4419999999999998e-06 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039486+0000", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0523999999999999e-05 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039502+0000", 2024-09-17T22:57:58.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5930000000000002e-05 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039590+0000", 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8012999999999999e-05 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047208+0000", 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0076180129999999999 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.078 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1121 3.4 e28/22 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head v 28'6, mlcod=27'4)", 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.015295162000001, 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004866296, 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.079 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042566+0000", 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:57:58.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042575+0000", 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3349999999999996e-06 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042580+0000", 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042623+0000", 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3396000000000003e-05 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042644+0000", 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1155999999999999e-05 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047379+0000", 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047352699999999998 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047426+0000", 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6424999999999997e-05 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.084 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6774 3.0 e33/22 3:05c44946:::rbd_data.118846a2cbc0.0000000000000045:head v 33'36, mlcod=32'33)", 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 26.982410115, 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053547780000000001, 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0299999999999998e-07 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075451+0000", 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:57:58.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075468+0000", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7212999999999999e-05 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075474+0000", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075522+0000", 2024-09-17T22:57:58.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8114e-05 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075543+0000", 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0389000000000001e-05 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080753+0000", 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052103440000000004 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080799+0000", 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6335999999999998e-05 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7337 3.11 e33/28 3:89c4a1e2:::rbd_header.118846a2cbc0:head v 33'39, mlcod=33'36)", 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 26.862566502, 2024-09-17T22:57:58.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046186769999999999, 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195289+0000", 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195293+0000", 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.155e-06 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195301+0000", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8520000000000004e-06 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195307+0000", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195340+0000", 2024-09-17T22:57:58.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3090999999999999e-05 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195353+0000", 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2757e-05 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199876+0000", 2024-09-17T22:57:58.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045229500000000004 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199907+0000", 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1340000000000001e-05 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.094 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22674 3.0 e45/22 3:000cde75:::rbd_data.1188903913c1.000000000000001f:head v 45'77, mlcod=44'74)", 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.917911085, 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0062289370000000004, 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139953+0000", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0229999999999998e-06 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139962+0000", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-06 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139968+0000", 2024-09-17T22:57:58.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140030+0000", 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1803000000000004e-05 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140046+0000", 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6093999999999999e-05 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146129+0000", 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0060834239999999996 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146173+0000", 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3238999999999997e-05 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.0 epoch 45/22 rep_tid 301 entries 45'78 (0'0) error 3:047d1ef5:::rbd_data.1188903913c1.0000000000000029:head by client.4488.0:22684 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 44'74)", 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:58.099 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.916212847000001, 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.007471218, 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.44e-07 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141651+0000", 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4449999999999996e-06 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141671+0000", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0373e-05 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141695+0000", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4224e-05 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141782+0000", 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6514999999999994e-05 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149113+0000", 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0073311169999999998 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22687 3.4 e45/22 3:25296614:::rbd_data.1188903913c1.000000000000002b:head v 45'156, mlcod=44'152)", 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.915339232999999, 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0066082959999999996, 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.103 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142516+0000", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3900000000000004e-07 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142523+0000", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7429999999999996e-06 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142536+0000", 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3151e-05 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142543+0000", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7100000000000001e-06 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142607+0000", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3765999999999999e-05 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142627+0000", 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.035e-05 2024-09-17T22:57:58.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149088+0000", 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0064607140000000002 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149124+0000", 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6123e-05 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22907 3.4 e45/22 3:244fb18a:::rbd_data.1188903913c1.00000000000001f6:head v 45'158, mlcod=45'154)", 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.89876632, 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031838320000000002, 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159089+0000", 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5499999999999998e-07 2024-09-17T22:57:58.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159095+0000", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4409999999999998e-06 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159108+0000", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.258e-05 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159115+0000", 2024-09-17T22:57:58.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9419999999999998e-06 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159210+0000", 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4853999999999997e-05 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159231+0000", 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1583e-05 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162234+0000", 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030021539999999999 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162272+0000", 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8622999999999999e-05 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:39761 3.4 e54/22 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:head v 54'167, mlcod=54'165)", 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:58.112 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.2388228449999996, 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031656729999999999, 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819033+0000", 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.562e-06 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819756+0000", 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00072241600000000003 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819777+0000", 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1429000000000001e-05 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819787+0000", 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0137e-05 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819890+0000", 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102441 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819913+0000", 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3529000000000001e-05 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822139+0000", 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022256530000000002 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822198+0000", 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8505999999999999e-05 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:42197 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:58.117 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.012411191, 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032112489999999998, 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.118 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 42197 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:57:58.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045445+0000", 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.615e-06 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045452+0000", 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8589999999999999e-06 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045463+0000", 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1001e-05 2024-09-17T22:57:58.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045481+0000", 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8423e-05 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045537+0000", 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6069999999999997e-05 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045626+0000", 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8351999999999997e-05 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048535+0000", 2024-09-17T22:57:58.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029092060000000001 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048589+0000", 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3761999999999998e-05 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048655+0000", 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5961000000000001e-05 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.126 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44169 3.11 3:8b5f814e:::rbd_data.118894cf7479.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.821967092, 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0042738230000000004, 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.127 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44169 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235890+0000", 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0789999999999999e-06 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235900+0000", 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0277e-05 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235923+0000", 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2693000000000001e-05 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235933+0000", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0441e-05 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236005+0000", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1957000000000002e-05 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236091+0000", 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5718999999999997e-05 2024-09-17T22:57:58.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240059+0000", 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0039685099999999997 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240096+0000", 2024-09-17T22:57:58.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6431999999999999e-05 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240161+0000", 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5715000000000004e-05 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44203 3.0 e54/22 3:037ebc53:::rbd_data.118894cf7479.000000000000003a:head v 54'130, mlcod=54'128)", 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.785890958, 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0035646229999999998, 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271965+0000", 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1739999999999999e-06 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271973+0000", 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8650000000000001e-06 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271986+0000", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2899e-05 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271993+0000", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2520000000000002e-06 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272061+0000", 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392000000000005e-05 2024-09-17T22:57:58.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272083+0000", 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2161e-05 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275498+0000", 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034148059999999998 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275528+0000", 2024-09-17T22:57:58.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0074e-05 2024-09-17T22:57:58.138 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.138 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.138 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44204 3.4 e54/22 3:23e33dd1:::rbd_data.118894cf7479.000000000000003b:head v 54'172, mlcod=54'170)", 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.7858257559999999, 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034598849999999998, 2024-09-17T22:57:58.139 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.140 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.140 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.140 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.01e-07 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272034+0000", 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6179999999999997e-06 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272041+0000", 2024-09-17T22:57:58.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4290000000000004e-06 2024-09-17T22:57:58.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272047+0000", 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3890000000000004e-06 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272096+0000", 2024-09-17T22:57:58.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9033000000000003e-05 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272114+0000", 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8158000000000001e-05 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275444+0000", 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033300489999999999 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275489+0000", 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4808000000000002e-05 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.147 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44658 3.11 3:8a68e66c:::rbd_data.118894cf7479.000000000000007f:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.619532575, 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033015309999999999, 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.148 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44658 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:57:58.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438324+0000", 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.153e-06 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438332+0000", 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7759999999999997e-06 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438349+0000", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7005999999999999e-05 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438359+0000", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0186999999999999e-05 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438420+0000", 2024-09-17T22:57:58.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0915000000000002e-05 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438517+0000", 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6848999999999994e-05 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441513+0000", 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002996383 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441545+0000", 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1656999999999999e-05 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441624+0000", 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8604999999999997e-05 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.153 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:45020 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.363671955, 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0036221339999999999, 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.154 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 45020 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694185+0000", 2024-09-17T22:57:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694195+0000", 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0603e-05 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694222+0000", 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6573000000000001e-05 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694236+0000", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3816e-05 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694368+0000", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013227200000000001 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694508+0000", 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014027700000000001 2024-09-17T22:57:58.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697713+0000", 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032041769999999999 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697749+0000", 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6742999999999997e-05 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697805+0000", 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5535999999999998e-05 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.160 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48071 3.11 3:8902b897:::rbd_data.1188a43cbaa8.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.19469687099999999, 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.017833759000000001, 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48071 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.162 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.163 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863159+0000", 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2240000000000001e-06 2024-09-17T22:57:58.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863166+0000", 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5989999999999998e-06 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863181+0000", 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5637e-05 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863189+0000", 2024-09-17T22:57:58.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0360000000000006e-06 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863256+0000", 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6255999999999997e-05 2024-09-17T22:57:58.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863338+0000", 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2157000000000006e-05 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880830+0000", 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.017492245 2024-09-17T22:57:58.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880895+0000", 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4717999999999999e-05 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880992+0000", 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6886999999999996e-05 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48127 3.11 3:8902b897:::rbd_data.1188a43cbaa8.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write 2850816~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.122966308, 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048067099999999996, 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48127 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934890+0000", 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:57:58.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934914+0000", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3643999999999999e-05 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934927+0000", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3159000000000001e-05 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934933+0000", 2024-09-17T22:57:58.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8510000000000004e-06 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934987+0000", 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4497999999999999e-05 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.935080+0000", 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2577000000000005e-05 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:57:58.173 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939562+0000", 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0044823440000000001 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939620+0000", 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8019000000000001e-05 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939695+0000", 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4688e-05 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.174 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.175 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.175 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.175 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:58.176 INFO:teuthology.orchestra.run.smithi159.stdout:3108 write 0xa7636e1 thru 0xa76c06a (0x898a bytes) 2024-09-17T22:57:58.177 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3109 trunc from 0xa76c06b to 0x232020a 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3110 punch from 0x15f4cb2 to 0x15fa492, (0x57e0 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3111 write 0x24771ad thru 0x2479011 (0x1e65 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3112 read 0x1fc52cb thru 0x1fcf228 (0x9f5e bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3113 write 0xc64c0f4 thru 0xc652769 (0x6676 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3114 write 0x5453700 thru 0x5461c80 (0xe581 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3118 read 0x54c6784 thru 0x54ccd2b (0x65a8 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:3119 read 0x6249842 thru 0x6255f61 (0xc720 bytes) 2024-09-17T22:57:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3121 punch from 0x7adce94 to 0x7ae4543, (0x76af bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3125 write 0x1f0ebd2 thru 0x1f103a6 (0x17d5 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3127 write 0xc75d3c8 thru 0xc76a7fb (0xd434 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3128 write 0x4335eb6 thru 0x4341890 (0xb9db bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3129 read 0xcbb100 thru 0xcc5b80 (0xaa81 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3130 punch from 0x8717a5e to 0x871c824, (0x4dc6 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3132 read 0x8ecb937 thru 0x8eccc5e (0x1328 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3133 punch from 0x892b0c7 to 0x892ba24, (0x95d bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3135 write 0x859c2a4 thru 0x85a5324 (0x9081 bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3136 punch from 0xa4bc991 to 0xa4bcd7b, (0x3ea bytes) 2024-09-17T22:57:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:3137 punch from 0x5cde4b6 to 0x5ce4f32, (0x6a7c bytes) 2024-09-17T22:57:58.181 INFO:teuthology.orchestra.run.smithi159.stdout:3138 writesame 0x11d209f thru 0x11d4c08 data_size 0x1(0x2b6a bytes) 2024-09-17T22:57:58.181 INFO:teuthology.orchestra.run.smithi159.stdout:3139 write 0x59d913 thru 0x5a7bff (0xa2ed bytes) 2024-09-17T22:57:58.181 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:57:58.181 INFO:teuthology.orchestra.run.smithi159.stdout:3140 trunc from 0xc76a7fc to 0x8fd73a9 2024-09-17T22:57:58.182 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.182 INFO:teuthology.orchestra.run.smithi159.stdout:3143 writesame 0xa703d10 thru 0xa710e8f data_size 0x40(0xd180 bytes) 2024-09-17T22:57:58.183 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.183 INFO:teuthology.orchestra.run.smithi159.stdout:3145 read 0x6cf212a thru 0x6cf4deb (0x2cc2 bytes) 2024-09-17T22:57:58.183 INFO:teuthology.orchestra.run.smithi159.stdout:3148 punch from 0x6f42f4d to 0x6f4c615, (0x96c8 bytes) 2024-09-17T22:57:58.183 INFO:teuthology.orchestra.run.smithi159.stdout:3150 write 0x729152f thru 0x72a0cf2 (0xf7c4 bytes) 2024-09-17T22:57:58.183 INFO:teuthology.orchestra.run.smithi159.stdout:3151 writesame 0x8f4708a thru 0x8f5638f data_size 0x3(0xf306 bytes) 2024-09-17T22:57:58.183 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:58.174+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:58.185 INFO:teuthology.orchestra.run.smithi159.stdout:3155 write 0x8a6b475 thru 0x8a7822a (0xcdb6 bytes) 2024-09-17T22:57:58.194 INFO:teuthology.orchestra.run.smithi159.stdout:3156 writesame 0xbc9f572 thru 0xbca60f2 data_size 0x1d(0x6b81 bytes) 2024-09-17T22:57:58.197 INFO:teuthology.orchestra.run.smithi159.stdout:3157 trunc from 0xbca60f3 to 0x87d9f0f 2024-09-17T22:57:58.205 INFO:teuthology.orchestra.run.smithi159.stdout:3159 punch from 0x1cdf2b to 0x1d15d7, (0x36ac bytes) 2024-09-17T22:57:58.205 INFO:teuthology.orchestra.run.smithi159.stdout:3161 punch from 0x4f52fb8 to 0x4f617ea, (0xe832 bytes) 2024-09-17T22:57:58.205 INFO:teuthology.orchestra.run.smithi159.stdout:3162 write 0x1d42d8e thru 0x1d4b4a3 (0x8716 bytes) 2024-09-17T22:57:58.206 INFO:teuthology.orchestra.run.smithi159.stdout:3163 punch from 0x5d06a18 to 0x5d0eb77, (0x815f bytes) 2024-09-17T22:57:58.206 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.207 INFO:teuthology.orchestra.run.smithi159.stdout:3165 write 0x143bdd6 thru 0x1447caf (0xbeda bytes) 2024-09-17T22:57:58.208 INFO:teuthology.orchestra.run.smithi159.stdout:3166 read 0x3d182ec thru 0x3d2001f (0x7d34 bytes) 2024-09-17T22:57:58.208 INFO:teuthology.orchestra.run.smithi159.stdout:3167 punch from 0x73bab02 to 0x73c3969, (0x8e67 bytes) 2024-09-17T22:57:58.208 INFO:teuthology.orchestra.run.smithi159.stdout:3169 read 0x2d35485 thru 0x2d42dff (0xd97b bytes) 2024-09-17T22:57:58.209 INFO:teuthology.orchestra.run.smithi159.stdout:3170 read 0x2abdda6 thru 0x2acc482 (0xe6dd bytes) 2024-09-17T22:57:58.209 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.216 INFO:teuthology.orchestra.run.smithi159.stdout:3172 trunc from 0x87d9f0f to 0xe49b482 2024-09-17T22:57:58.221 INFO:teuthology.orchestra.run.smithi159.stdout:3174 read 0xa677c0e thru 0xa687bdd (0xffd0 bytes) 2024-09-17T22:57:58.228 INFO:teuthology.orchestra.run.smithi159.stdout:3175 trunc from 0xe49b482 to 0x8e7269f 2024-09-17T22:57:58.235 INFO:teuthology.orchestra.run.smithi159.stdout:3176 punch from 0x8c03da6 to 0x8c0bdfa, (0x8054 bytes) 2024-09-17T22:57:58.242 INFO:teuthology.orchestra.run.smithi159.stdout:3177 trunc from 0x8e7269f to 0x30d4a4e 2024-09-17T22:57:58.260 INFO:teuthology.orchestra.run.smithi159.stdout:3178 punch from 0x2d0df48 to 0x2d12eb5, (0x4f6d bytes) 2024-09-17T22:57:58.267 INFO:teuthology.orchestra.run.smithi159.stdout:3180 writesame 0x41e245b thru 0x41f1c68 data_size 0x1(0xf80e bytes) 2024-09-17T22:57:58.268 INFO:teuthology.orchestra.run.smithi159.stdout:3181 read 0x2163ab8 thru 0x216e82a (0xad73 bytes) 2024-09-17T22:57:58.275 INFO:teuthology.orchestra.run.smithi159.stdout:3182 write 0x5cbd188 thru 0x5cc2bd1 (0x5a4a bytes) 2024-09-17T22:57:58.276 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:58.274+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:58.279 INFO:teuthology.orchestra.run.smithi159.stdout:3184 flatten 2024-09-17T22:57:58.284 INFO:teuthology.orchestra.run.smithi159.stdout:3186 writesame 0x2089c12 thru 0x208b37f data_size 0x1(0x176e bytes) 2024-09-17T22:57:58.291 INFO:teuthology.orchestra.run.smithi159.stdout:3187 trunc from 0x5cc2bd2 to 0x26cac70 2024-09-17T22:57:58.305 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:57:58.305 INFO:teuthology.orchestra.run.smithi159.stdout:3190 clone 25 order 18 su 16384 sc 9 2024-09-17T22:57:58.313 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:58.314 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:58.314 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:57:58.314 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:58.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:57:58.377 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:58.375+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:58.466 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:57:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:57:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:57:58.466 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:58.474 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:57:58.518 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:58.516+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:57:58.579 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5757 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990059323000001, 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038199444999999999, 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.580 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5757 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5600000000000002e-07 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586464+0000", 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:57:58.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586468+0000", 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8630000000000001e-06 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586471+0000", 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7549999999999999e-06 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624560+0000", 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038088960999999998 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624607+0000", 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6740000000000003e-05 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624661+0000", 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4732999999999999e-05 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.584 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5758 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990046434, 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038315071999999999, 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5758 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5100000000000001e-07 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586476+0000", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.113e-06 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586483+0000", 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5200000000000003e-06 2024-09-17T22:57:58.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586487+0000", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3479999999999997e-06 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624681+0000", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038194371999999997 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624722+0000", 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0367999999999997e-05 2024-09-17T22:57:58.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624790+0000", 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8100000000000002e-05 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5759 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3948544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990030708999999, 2024-09-17T22:57:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038450864000000001, 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5759 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586491+0000", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586492+0000", 2024-09-17T22:57:58.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5349999999999999e-06 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586495+0000", 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586498+0000", 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6170000000000001e-06 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624828+0000", 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038329514000000002 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624875+0000", 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7107999999999997e-05 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624941+0000", 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6647000000000001e-05 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.593 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5760 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990020002000001, 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038669935000000003, 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5760 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586503+0000", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0899999999999999e-06 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586506+0000", 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:57:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586508+0000", 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624980+0000", 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038472343999999999 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625076+0000", 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5214e-05 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625171+0000", 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5421999999999995e-05 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5761 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4276224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.598 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990010022, 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038805352000000001, 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5761 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.22e-07 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586512+0000", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0210000000000001e-06 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586515+0000", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586518+0000", 2024-09-17T22:57:58.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625211+0000", 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038693268000000003 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625261+0000", 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9345000000000003e-05 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625317+0000", 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5850999999999998e-05 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5762 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4472832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.990000130999999, 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038916981000000003, 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.603 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5762 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586525+0000", 2024-09-17T22:57:58.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.224e-06 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586532+0000", 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3950000000000003e-06 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586535+0000", 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625339+0000", 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038804584000000003 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625385+0000", 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5099000000000003e-05 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625438+0000", 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3511999999999999e-05 2024-09-17T22:57:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5763 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4669440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.989982201, 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039029253999999999, 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.608 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5763 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:58.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586541+0000", 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.268e-06 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586547+0000", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586554+0000", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625465+0000", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038910924 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625511+0000", 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5179999999999998e-05 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625568+0000", 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7738999999999997e-05 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5764 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4866048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:58.612 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.989961350000002, 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039122912000000003, 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5764 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586561+0000", 2024-09-17T22:57:58.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.266e-06 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586566+0000", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586568+0000", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625592+0000", 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039024174000000002 2024-09-17T22:57:58.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625634+0000", 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1464000000000001e-05 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625683+0000", 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9002000000000003e-05 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5765 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5062656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.989949502999998, 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039240865999999999, 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5765 2024-09-17T22:57:58.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586895+0000", 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000323658 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586899+0000", 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6260000000000001e-06 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586910+0000", 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1239e-05 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586917+0000", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625710+0000", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038792608999999999 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625757+0000", 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7416000000000001e-05 2024-09-17T22:57:58.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625813+0000", 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5395000000000001e-05 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5766 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5259264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:58.621 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.989593371000002, 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038992026999999999, 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5766 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586931+0000", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586939+0000", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2549999999999997e-06 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586945+0000", 2024-09-17T22:57:58.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625828+0000", 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038882947000000001 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625871+0000", 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3204000000000002e-05 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625920+0000", 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8539999999999999e-05 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5767 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5455872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.989568281, 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039097260000000002, 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.626 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5767 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:57:58.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588020+0000", 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00106675 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588024+0000", 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588033+0000", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588044+0000", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1800000000000001e-05 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625943+0000", 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037898695000000003 2024-09-17T22:57:58.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625988+0000", 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4577999999999999e-05 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626050+0000", 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2605000000000001e-05 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5769 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5849088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.988437649000002, 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038207729000000003, 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5769 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.631 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588086+0000", 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.103e-06 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588094+0000", 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6459999999999993e-06 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588099+0000", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1259999999999997e-06 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626190+0000", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.03809067 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626234+0000", 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4051999999999998e-05 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626291+0000", 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7694e-05 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5770 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.988414677000002, 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038303865999999999, 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.635 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5770 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588109+0000", 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0899999999999999e-06 2024-09-17T22:57:58.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588116+0000", 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2439999999999996e-06 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588122+0000", 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8640000000000001e-06 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626315+0000", 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038193105999999998 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626359+0000", 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3828000000000001e-05 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626410+0000", 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1328000000000001e-05 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.639 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5771 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.988392396999998, 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038402700999999997, 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.640 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5771 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588131+0000", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588138+0000", 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3049999999999998e-06 2024-09-17T22:57:58.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588143+0000", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4650000000000002e-06 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626433+0000", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038289895999999997 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626476+0000", 2024-09-17T22:57:58.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2913000000000001e-05 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626532+0000", 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5692000000000001e-05 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5772 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6438912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.988370739, 2024-09-17T22:57:58.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038494234000000002, 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5772 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590017+0000", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018664980000000001 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590020+0000", 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4769999999999999e-06 2024-09-17T22:57:58.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590027+0000", 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5209999999999999e-06 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590032+0000", 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626554+0000", 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036521551999999999 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626595+0000", 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1900000000000002e-05 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626645+0000", 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9205000000000001e-05 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.648 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5828 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.986177332, 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038160161999999997, 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.649 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5828 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590344+0000", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4600000000000001e-07 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590345+0000", 2024-09-17T22:57:58.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.376e-06 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590349+0000", 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590352+0000", 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7410000000000001e-06 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628400+0000", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038048347000000003 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628448+0000", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7961999999999999e-05 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628504+0000", 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6137000000000001e-05 2024-09-17T22:57:58.653 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5829 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.986166276999999, 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038295726000000002, 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5829 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592028+0000", 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001673379 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592032+0000", 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5109999999999999e-06 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592038+0000", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7379999999999999e-06 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592042+0000", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628543+0000", 2024-09-17T22:57:58.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036500828999999999 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628594+0000", 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1737999999999998e-05 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628651+0000", 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6320999999999998e-05 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5840 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.984315641999999, 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038232347, 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5840 2024-09-17T22:57:58.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592208+0000", 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-06 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592214+0000", 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7479999999999999e-06 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592220+0000", 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.037e-06 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630310+0000", 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038090474999999999 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630373+0000", 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2467999999999999e-05 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630438+0000", 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5235999999999997e-05 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.663 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5841 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3375104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.984295404000001, 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038361579999999999, 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5841 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:57:58.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592228+0000", 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.666 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592235+0000", 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8040000000000004e-06 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592242+0000", 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2230000000000001e-06 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630468+0000", 2024-09-17T22:57:58.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038225982999999998 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630517+0000", 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8656000000000003e-05 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630587+0000", 2024-09-17T22:57:58.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0698000000000006e-05 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5842 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.984272674, 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038469774999999998, 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:57:58.669 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5842 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:57:58.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594021+0000", 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001772849 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594026+0000", 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4399999999999998e-06 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594037+0000", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1071e-05 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594044+0000", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9129999999999997e-06 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630613+0000", 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036569602999999999 2024-09-17T22:57:58.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630659+0000", 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5246e-05 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630718+0000", 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9653e-05 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:57:58.673 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:57:58.674 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:57:58.674 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:58.576+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:58.675 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:57:58.675 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-17T22:57:58.675 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:57:58.675 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:57:58.676 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-17T22:57:58.719 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:58.717+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:58.781 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:58.781 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:58.781 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:58.781 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:58.781 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:58.778+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:58.789 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:57:58.882 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:57:58.879+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:57:58.932 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:58.932 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:58.933 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:58.933 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:58.933 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:58.942 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:57:58.997 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:58.994+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:59.001 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-09-17T22:57:58.999+0000 7fe9a1abd640 -1 mon.a@0(leader).osd e58 definitely_dead 0 2024-09-17T22:57:59.053 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24281 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.817068037, 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037645445, 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.054 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24281 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232602+0000", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232605+0000", 2024-09-17T22:57:59.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-06 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232612+0000", 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.596e-06 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232623+0000", 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.269238+0000", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036615385 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270084+0000", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084551799999999996 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270247+0000", 2024-09-17T22:57:59.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000162948 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24284 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.817039558999999, 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037880098000000001, 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.059 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24284 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232633+0000", 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4320000000000002e-06 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232639+0000", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4389999999999997e-06 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232644+0000", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.651e-06 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270285+0000", 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037641711000000001 2024-09-17T22:57:59.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270368+0000", 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2358e-05 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270510+0000", 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014204899999999999 2024-09-17T22:57:59.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24285 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1966080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.817019397999999, 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038129107000000002, 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.064 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24285 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232652+0000", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1150000000000001e-06 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232659+0000", 2024-09-17T22:57:59.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232663+0000", 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.403e-06 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270554+0000", 2024-09-17T22:57:59.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037890946000000002 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270636+0000", 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1919e-05 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270779+0000", 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000143103 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24286 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.817000168, 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038334057999999997, 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24286 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232677+0000", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1539999999999996e-06 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232683+0000", 2024-09-17T22:57:59.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8739999999999997e-06 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232689+0000", 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3299999999999998e-06 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270820+0000", 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038131089999999999 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270888+0000", 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8288999999999996e-05 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271003+0000", 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114955 2024-09-17T22:57:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24289 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816973424, 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038485336000000002, 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.074 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24289 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232698+0000", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0320000000000002e-06 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232704+0000", 2024-09-17T22:57:59.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232713+0000", 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2930000000000006e-06 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271036+0000", 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038323004000000001 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271078+0000", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1063999999999999e-05 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271181+0000", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103559 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24290 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:59.078 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816949179, 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038652381, 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24290 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:59.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232722+0000", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.561e-06 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232727+0000", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6080000000000003e-06 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232732+0000", 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4120000000000003e-06 2024-09-17T22:57:59.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271214+0000", 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038481808999999999 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271268+0000", 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4561999999999997e-05 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271372+0000", 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104101 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24293 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816931351000001, 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038796900000000002, 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.083 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24293 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232740+0000", 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232744+0000", 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232747+0000", 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.689e-06 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271401+0000", 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038653976999999999 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271442+0000", 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0547999999999999e-05 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271535+0000", 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3034000000000004e-05 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24294 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816918698, 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039269964999999997, 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.088 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24294 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232751+0000", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232752+0000", 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1829999999999999e-06 2024-09-17T22:57:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232755+0000", 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1860000000000001e-06 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232757+0000", 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000001e-06 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271598+0000", 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038840787000000002 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271794+0000", 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000195434 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272020+0000", 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022673999999999999 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.092 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24369 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1081344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816533697000001, 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039126368000000002, 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.093 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24369 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233136+0000", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2099999999999995e-07 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233141+0000", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6380000000000003e-06 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233152+0000", 2024-09-17T22:57:59.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1405e-05 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233160+0000", 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272065+0000", 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905397000000001 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272141+0000", 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5465999999999999e-05 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272262+0000", 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000120999 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24370 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816496218999999, 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039291936999999999, 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24370 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:57:59.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3900000000000005e-07 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233176+0000", 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233188+0000", 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1535999999999999e-05 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233204+0000", 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5678000000000001e-05 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272290+0000", 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039085904999999997 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272356+0000", 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6587999999999993e-05 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272465+0000", 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108724 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.102 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24373 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816455149999999, 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039461167999999998, 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.103 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24373 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233215+0000", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233217+0000", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233224+0000", 2024-09-17T22:57:59.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9739999999999996e-06 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233230+0000", 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4809999999999997e-06 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272491+0000", 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039261457 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272560+0000", 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8576000000000002e-05 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272675+0000", 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011552699999999999 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24374 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:59.107 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816427089999999, 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039645639000000003, 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24374 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233243+0000", 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233245+0000", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6000000000000001e-06 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233252+0000", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9650000000000002e-06 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233257+0000", 2024-09-17T22:57:59.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2549999999999997e-06 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272704+0000", 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039446390999999997 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272778+0000", 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3818000000000004e-05 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272888+0000", 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110019 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24377 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1097728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816403821, 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039817298000000001, 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24377 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233266+0000", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233268+0000", 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:57:59.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233275+0000", 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6000000000000003e-06 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233279+0000", 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272918+0000", 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039638698 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272980+0000", 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1463000000000004e-05 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273083+0000", 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010304799999999999 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24378 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816382621000001, 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039979973000000002, 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24378 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233289+0000", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233295+0000", 2024-09-17T22:57:59.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0970000000000002e-06 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233300+0000", 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7890000000000002e-06 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273107+0000", 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039806766 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273167+0000", 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0541999999999997e-05 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273267+0000", 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9185999999999995e-05 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.121 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24381 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816361498999999, 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040144735000000001, 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24381 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:57:59.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233613+0000", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030540199999999997 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233623+0000", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.166e-06 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233630+0000", 2024-09-17T22:57:59.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8870000000000003e-06 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273292+0000", 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039662289000000003 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273352+0000", 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0600999999999998e-05 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273452+0000", 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9982999999999995e-05 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24382 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1228800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.816026749000001, 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040027144000000001, 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.126 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24382 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:57:59.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233643+0000", 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233647+0000", 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8210000000000003e-06 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233654+0000", 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3259999999999998e-06 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233660+0000", 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3210000000000004e-06 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273478+0000", 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039818781999999997 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273539+0000", 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0214999999999999e-05 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273670+0000", 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013101599999999999 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.130 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24385 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.815691891, 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039903028, 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24385 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233978+0000", 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:57:59.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233989+0000", 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0685e-05 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234000+0000", 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1282e-05 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234007+0000", 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273697+0000", 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039690053000000003 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273766+0000", 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8483000000000001e-05 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273880+0000", 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011480300000000001 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24386 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.815648641999999, 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040055687, 2024-09-17T22:57:59.135 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24386 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234024+0000", 2024-09-17T22:57:59.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234035+0000", 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1319e-05 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234041+0000", 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8170000000000004e-06 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273907+0000", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039866255000000003 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273972+0000", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4966000000000001e-05 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274076+0000", 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104293 2024-09-17T22:57:59.139 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24390 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1122304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.815619769, 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040224370000000002, 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.140 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24390 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:57:59.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234050+0000", 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234052+0000", 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.278e-06 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234059+0000", 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234381+0000", 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032158499999999999 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274100+0000", 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039719415000000001 2024-09-17T22:57:59.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274163+0000", 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3095000000000005e-05 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274274+0000", 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110373 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24391 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.815275381999999, 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040238796, 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24391 2024-09-17T22:57:59.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234395+0000", 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1800000000000005e-07 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234398+0000", 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6839999999999998e-06 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234408+0000", 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-05 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234415+0000", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274457+0000", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040042021999999997 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274528+0000", 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1940999999999994e-05 2024-09-17T22:57:59.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274633+0000", 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104114 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.149 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.151 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:57:59.151 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:57:59.148+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:57:59.152 INFO:teuthology.orchestra.run.smithi159.stderr:marked out osd.5. 2024-09-17T22:57:59.251 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:59.249+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:59.295 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.296 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:59.296 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:59.296 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:57:59.389 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:59.387+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:59.448 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.448 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:59.448 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:59.448 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:59.448 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.452 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:59.450+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:59.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:57:59.554 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:59.551+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:59.601 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.601 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:57:59.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:57:59.601 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:57:59.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345297211999998, 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:57:59.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:57:59.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345281206999999, 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:57:59.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:57:59.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:57:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:59.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345248436999999, 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:57:59.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345220698999999, 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:57:59.616 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:59.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:57:59.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345198280000002, 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.621 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:59.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.345175999999999, 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:57:59.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.344968595000001, 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:57:59.630 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:59.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:57:59.634 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.34442198, 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:57:59.635 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:57:59.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.343350783999998, 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:57:59.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.644 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.343339328999999, 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.645 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:57:59.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.343328352, 2024-09-17T22:57:59.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:59.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:57:59.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:57:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.34125422, 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:57:59.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.341241906, 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.659 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:57:59.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.341228668999999, 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:59.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.341217839999999, 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:57:59.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:57:59.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:57:59.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.339258964999999, 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:57:59.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:57:59.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:57:59.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:57:59.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.339234607000002, 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.677 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:57:59.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:57:59.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.339205389, 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:57:59.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.338438821, 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.686 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:57:59.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:57:59.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.338425719, 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:59.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:57:59.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:57:59.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:57:59.695 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.696 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:57:59.697 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:57:59.694+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:57:59.797 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:57:59.795+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:57:59.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:57:59.841 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:59.841 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:57:59.841 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.849 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:57:59.933 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:57:59.931+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:57:59.975 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:57:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:57:59.992 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:57:59.992 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:57:59.993 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:57:59.993 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:57:59.994 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:57:59.994 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:57:59.994 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:57:59.998 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:57:59.995+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:00.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:00.008 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:00.008 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:00.024 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:00.025 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:00.040 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:00.041 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:00.041 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:00.057 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:00.057 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:00.073 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:00.073 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:00.076 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:00.077 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:00.099 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:00.097+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:00.146 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.146 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.155 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:00.183 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:00.200 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:00.197+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:00.261 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.261 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.270 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:00.313 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:00.311+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:00.378 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.378 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.387 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:00.401 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:00.399+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:00.503 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:00.501+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:00.531 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.531 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:00.604 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:00.601+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:00.642 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.643 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.651 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:00.735 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:00.733+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:00.795 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:00.795 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:00.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:00.808 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:00.803+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:00.849 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:58:00.912 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:00.910+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:00.987 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T22:58:01.004 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.004 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.008 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:01.005+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:01.022 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:01.108 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:01.106+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:01.177 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.178 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:01.229 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:01.226+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:01.258 INFO:teuthology.orchestra.run.smithi159.stdout:3191 read 0xa6a35d thru 0xa71cba (0x795e bytes) 2024-09-17T22:58:01.259 INFO:teuthology.orchestra.run.smithi159.stdout:3192 punch from 0x25c7330 to 0x25d7155, (0xfe25 bytes) 2024-09-17T22:58:01.262 INFO:teuthology.orchestra.run.smithi159.stdout:3193 punch from 0x1f97381 to 0x1f9d5b2, (0x6231 bytes) 2024-09-17T22:58:01.269 INFO:teuthology.orchestra.run.smithi159.stdout:3194 write 0x3d69e8f thru 0x3d78817 (0xe989 bytes) 2024-09-17T22:58:01.280 INFO:teuthology.orchestra.run.smithi159.stdout:3195 read 0x20fcdb1 thru 0x20fd0c3 (0x313 bytes) 2024-09-17T22:58:01.280 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.280 INFO:teuthology.orchestra.run.smithi159.stdout:3200 punch from 0x2001de8 to 0x2009de1, (0x7ff9 bytes) 2024-09-17T22:58:01.288 INFO:teuthology.orchestra.run.smithi159.stdout:3201 write 0x48ea6ad thru 0x48f8478 (0xddcc bytes) 2024-09-17T22:58:01.305 INFO:teuthology.orchestra.run.smithi159.stdout:3203 write 0x52f3d82 thru 0x52f45be (0x83d bytes) 2024-09-17T22:58:01.308 INFO:teuthology.orchestra.run.smithi159.stdout:3204 punch from 0x529cf1a to 0x52aa986, (0xda6c bytes) 2024-09-17T22:58:01.308 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.309 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.309 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.311 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:01.309+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:01.321 INFO:teuthology.orchestra.run.smithi159.stdout:3210 writesame 0xb49737e thru 0xb4a621f data_size 0x95(0xeea2 bytes) 2024-09-17T22:58:01.321 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.321 INFO:teuthology.orchestra.run.smithi159.stdout:3212 punch from 0x483bcb to 0x484b7f, (0xfb4 bytes) 2024-09-17T22:58:01.321 INFO:teuthology.orchestra.run.smithi159.stdout:3213 punch from 0x1236178 to 0x1239420, (0x32a8 bytes) 2024-09-17T22:58:01.329 INFO:teuthology.orchestra.run.smithi159.stdout:3214 writesame 0xddda7f5 thru 0xdde481a data_size 0x3(0xa026 bytes) 2024-09-17T22:58:01.330 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.330 INFO:teuthology.orchestra.run.smithi159.stdout:3216 trunc from 0xdde481b to 0xdd96b62 2024-09-17T22:58:01.334 INFO:teuthology.orchestra.run.smithi159.stdout:3217 read 0x7603c8f thru 0x7610226 (0xc598 bytes) 2024-09-17T22:58:01.334 INFO:teuthology.orchestra.run.smithi159.stdout:3218 read 0x4064431 thru 0x40703bb (0xbf8b bytes) 2024-09-17T22:58:01.334 INFO:teuthology.orchestra.run.smithi159.stdout:3219 read 0x26a7578 thru 0x26b5568 (0xdff1 bytes) 2024-09-17T22:58:01.334 INFO:teuthology.orchestra.run.smithi159.stdout:3221 writesame 0x8b3e8bc thru 0x8b4497c data_size 0x1f(0x60c1 bytes) 2024-09-17T22:58:01.338 INFO:teuthology.orchestra.run.smithi159.stdout:3222 read 0x1b69529 thru 0x1b6e71e (0x51f6 bytes) 2024-09-17T22:58:01.339 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.339 INFO:teuthology.orchestra.run.smithi159.stdout:3224 read 0xbf6fc2b thru 0xbf7123e (0x1614 bytes) 2024-09-17T22:58:01.339 INFO:teuthology.orchestra.run.smithi159.stdout:3225 read 0x364bc44 thru 0x3657e9e (0xc25b bytes) 2024-09-17T22:58:01.339 INFO:teuthology.orchestra.run.smithi159.stdout:3228 writesame 0xcc2a22c thru 0xcc32256 data_size 0x1(0x802b bytes) 2024-09-17T22:58:01.346 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.346 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.349 INFO:teuthology.orchestra.run.smithi159.stdout:3230 writesame 0x54d51b5 thru 0x54e231a data_size 0x7(0xd166 bytes) 2024-09-17T22:58:01.355 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:01.358 INFO:teuthology.orchestra.run.smithi159.stdout:3232 punch from 0x653a4cb to 0x6541fe9, (0x7b1e bytes) 2024-09-17T22:58:01.358 INFO:teuthology.orchestra.run.smithi159.stdout:3234 write 0x8e09eea thru 0x8e15f46 (0xc05d bytes) 2024-09-17T22:58:01.367 INFO:teuthology.orchestra.run.smithi159.stdout:3235 write 0x5451215 thru 0x5451fe1 (0xdcd bytes) 2024-09-17T22:58:01.368 INFO:teuthology.orchestra.run.smithi159.stdout:3236 write 0x7d0c51 thru 0x7d9b5b (0x8f0b bytes) 2024-09-17T22:58:01.378 INFO:teuthology.orchestra.run.smithi159.stdout:3239 write 0x10a732f thru 0x10b3ddb (0xcaad bytes) 2024-09-17T22:58:01.390 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.390 INFO:teuthology.orchestra.run.smithi159.stdout:3241 write 0xb93af09 thru 0xb93e21b (0x3313 bytes) 2024-09-17T22:58:01.394 INFO:teuthology.orchestra.run.smithi159.stdout:3243 punch from 0x40499a6 to 0x404fb63, (0x61bd bytes) 2024-09-17T22:58:01.394 INFO:teuthology.orchestra.run.smithi159.stdout:3244 write 0xcf1cafc thru 0xcf2b256 (0xe75b bytes) 2024-09-17T22:58:01.399 INFO:teuthology.orchestra.run.smithi159.stdout:3246 read 0xd90060f thru 0xd90a23e (0x9c30 bytes) 2024-09-17T22:58:01.403 INFO:teuthology.orchestra.run.smithi159.stdout:3247 trunc from 0xdd96b62 to 0xa7a0a3a 2024-09-17T22:58:01.412 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:01.410+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:01.412 INFO:teuthology.orchestra.run.smithi159.stdout:3248 write 0x9a3c024 thru 0x9a41fc5 (0x5fa2 bytes) 2024-09-17T22:58:01.427 INFO:teuthology.orchestra.run.smithi159.stdout:3249 writesame 0xc905548 thru 0xc9096f9 data_size 0x3(0x41b2 bytes) 2024-09-17T22:58:01.427 INFO:teuthology.orchestra.run.smithi159.stdout:3250 writesame 0x3e06bc4 thru 0x3e13504 data_size 0x1(0xc941 bytes) 2024-09-17T22:58:01.439 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.439 INFO:teuthology.orchestra.run.smithi159.stdout:3253 write 0xb46beae thru 0xb46e932 (0x2a85 bytes) 2024-09-17T22:58:01.443 INFO:teuthology.orchestra.run.smithi159.stdout:3254 read 0x9e485f1 thru 0x9e4d516 (0x4f26 bytes) 2024-09-17T22:58:01.443 INFO:teuthology.orchestra.run.smithi159.stdout:3255 read 0x225609f thru 0x2258d36 (0x2c98 bytes) 2024-09-17T22:58:01.450 INFO:teuthology.orchestra.run.smithi159.stdout:3256 writesame 0xdf879d7 thru 0xdf8e847 data_size 0x7(0x6e71 bytes) 2024-09-17T22:58:01.451 INFO:teuthology.orchestra.run.smithi159.stdout:3257 read 0x96eea9b thru 0x96f5d38 (0x729e bytes) 2024-09-17T22:58:01.451 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.451 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.451 INFO:teuthology.orchestra.run.smithi159.stdout:3264 write 0x68fe25f thru 0x690653b (0x82dd bytes) 2024-09-17T22:58:01.477 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.477 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.485 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:01.526 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:01.524+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:01.614 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:01.612+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:01.630 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.630 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:01.655 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.655 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.656 INFO:teuthology.orchestra.run.smithi159.stdout:3269 read 0x50be4f4 thru 0x50c3779 (0x5286 bytes) 2024-09-17T22:58:01.656 INFO:teuthology.orchestra.run.smithi159.stdout:3270 trunc from 0xdf8e848 to 0xd01d6ae 2024-09-17T22:58:01.662 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.665 INFO:teuthology.orchestra.run.smithi159.stdout:3272 trunc from 0xd01d6ae to 0xa6cfe0e 2024-09-17T22:58:01.674 INFO:teuthology.orchestra.run.smithi159.stdout:3273 writesame 0x4669b2 thru 0x468b09 data_size 0x4(0x2158 bytes) 2024-09-17T22:58:01.680 INFO:teuthology.orchestra.run.smithi159.stdout:3275 write 0x5d313ae thru 0x5d3d4a7 (0xc0fa bytes) 2024-09-17T22:58:01.697 INFO:teuthology.orchestra.run.smithi159.stdout:3276 write 0xe616cbf thru 0xe626099 (0xf3db bytes) 2024-09-17T22:58:01.704 INFO:teuthology.orchestra.run.smithi159.stdout:3278 writesame 0x87d198f thru 0x87da8a3 data_size 0x1(0x8f15 bytes) 2024-09-17T22:58:01.715 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:01.712+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:01.721 INFO:teuthology.orchestra.run.smithi159.stdout:3279 read 0xa9c3dc1 thru 0xa9cdab7 (0x9cf7 bytes) 2024-09-17T22:58:01.736 INFO:teuthology.orchestra.run.smithi159.stdout:3281 trunc from 0xe62609a to 0x1a844a2 2024-09-17T22:58:01.771 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.771 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.784 INFO:teuthology.orchestra.run.smithi159.stdout:3282 trunc from 0x1a844a2 to 0xcc698ed 2024-09-17T22:58:01.786 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:3290 read 0xafb2466 thru 0xafb9326 (0x6ec1 bytes) 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:3291 punch from 0x5c079f5 to 0x5c14119, (0xc724 bytes) 2024-09-17T22:58:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:3292 write 0xbb83384 thru 0xbb914fe (0xe17b bytes) 2024-09-17T22:58:01.798 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.799 INFO:teuthology.orchestra.run.smithi159.stdout:3294 write 0x90a736f thru 0x90aed3b (0x79cd bytes) 2024-09-17T22:58:01.816 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:01.814+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:01.816 INFO:teuthology.orchestra.run.smithi159.stdout:3296 trunc from 0xcc698ed to 0x65b2667 2024-09-17T22:58:01.836 INFO:teuthology.orchestra.run.smithi159.stdout:3297 trunc from 0x65b2667 to 0xbeccef4 2024-09-17T22:58:01.841 INFO:teuthology.orchestra.run.smithi159.stdout:3298 write 0x926906d thru 0x9278b91 (0xfb25 bytes) 2024-09-17T22:58:01.865 INFO:teuthology.orchestra.run.smithi159.stdout:3299 writesame 0xd0a7351 thru 0xd0ac6cc data_size 0x27(0x537c bytes) 2024-09-17T22:58:01.866 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:01.874 INFO:teuthology.orchestra.run.smithi159.stdout:3301 write 0xebb116e thru 0xebb3034 (0x1ec7 bytes) 2024-09-17T22:58:01.906 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:01.906 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:01.914 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:01.916 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:01.914+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:02.017 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:02.015+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:02.019 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:02.019 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:02.028 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:02.078 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:02.121 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:02.119+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:02.131 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:02.131 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:02.140 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:02.197 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:02.197 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:02.213 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:02.219 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:02.216+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:02.229 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:02.229 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:02.244 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:02.245 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:02.245 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:02.245 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:02.252 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:02.261 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:02.262 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:02.262 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:02.278 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:02.278 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:02.296 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:02.296 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:02.299 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:02.299 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:02.340 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:02.338+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:02.357 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:02.357 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:02.357 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:02.365 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:02.422 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:02.419+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:02.469 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:02.470 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:02.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:02.523 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:02.521+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366945664999999, 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.623 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.36692966, 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.36689689, 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366869152, 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366846732999999, 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.645 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366824453, 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:02.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:02.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366617047999998, 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.650 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:02.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.657 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.366070433000001, 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:02.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.364999236999999, 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:02.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.666 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.364987782, 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:02.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:02.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.364976805000001, 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:02.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.362902673000001, 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:02.675 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:02.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:02.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.362890359000001, 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:02.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.362877122, 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.684 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.362866293, 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.689 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:02.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:02.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.360907418, 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:02.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:02.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.360883059999999, 2024-09-17T22:58:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:02.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.702 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.360853842000001, 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.703 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:02.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.707 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.360087274000001, 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.708 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.360074172000001, 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:02.712 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:02.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:02.716 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:02.717 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:02.718 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:02.718 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:02.716+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:02.818 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:02.817+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:02.864 INFO:teuthology.orchestra.run.smithi159.stdout:3302 read 0xbf9643 thru 0xc07323 (0xdce1 bytes) 2024-09-17T22:58:02.865 INFO:teuthology.orchestra.run.smithi159.stdout:3303 read 0xaef50cf thru 0xaefa6ba (0x55ec bytes) 2024-09-17T22:58:02.865 INFO:teuthology.orchestra.run.smithi159.stdout:3304 write 0x6e49bbb thru 0x6e4bb24 (0x1f6a bytes) 2024-09-17T22:58:02.865 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:02.866 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:02.868 INFO:teuthology.orchestra.run.smithi159.stdout:3305 punch from 0x404bbda to 0x4050056, (0x447c bytes) 2024-09-17T22:58:02.868 INFO:teuthology.orchestra.run.smithi159.stdout:3306 read 0x9eb0381 thru 0x9eb9770 (0x93f0 bytes) 2024-09-17T22:58:02.869 INFO:teuthology.orchestra.run.smithi159.stdout:3308 writesame 0x56098ce thru 0x560a437 data_size 0x3(0xb6a bytes) 2024-09-17T22:58:02.871 INFO:teuthology.orchestra.run.smithi159.stdout:3311 punch from 0x7e87b93 to 0x7e977c6, (0xfc33 bytes) 2024-09-17T22:58:02.871 INFO:teuthology.orchestra.run.smithi159.stdout:3312 write 0x9338369 thru 0x934475b (0xc3f3 bytes) 2024-09-17T22:58:02.873 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:58:02.876 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:02.896 INFO:teuthology.orchestra.run.smithi159.stdout:3313 trunc from 0xebb3035 to 0x19d21b3 2024-09-17T22:58:02.910 INFO:teuthology.orchestra.run.smithi159.stdout:3314 write 0x1800aa1 thru 0x1810202 (0xf762 bytes) 2024-09-17T22:58:02.921 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:02.918+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:02.941 INFO:teuthology.orchestra.run.smithi159.stdout:3315 write 0xe31724f thru 0xe319425 (0x21d7 bytes) 2024-09-17T22:58:02.945 INFO:teuthology.orchestra.run.smithi159.stdout:3318 punch from 0x25fcb5d to 0x2609bc6, (0xd069 bytes) 2024-09-17T22:58:02.945 INFO:teuthology.orchestra.run.smithi159.stdout:3319 writesame 0xd377818 thru 0xd38572c data_size 0x17(0xdf15 bytes) 2024-09-17T22:58:02.951 INFO:teuthology.orchestra.run.smithi159.stdout:3321 read 0x8ce1cfb thru 0x8cecda1 (0xb0a7 bytes) 2024-09-17T22:58:02.951 INFO:teuthology.orchestra.run.smithi159.stdout:3322 write 0x128eb37 thru 0x1298924 (0x9dee bytes) 2024-09-17T22:58:02.961 INFO:teuthology.orchestra.run.smithi159.stdout:3323 read 0x3dcdb87 thru 0x3dd0bb6 (0x3030 bytes) 2024-09-17T22:58:02.961 INFO:teuthology.orchestra.run.smithi159.stdout:3324 read 0x1646c66 thru 0x1656bba (0xff55 bytes) 2024-09-17T22:58:02.961 INFO:teuthology.orchestra.run.smithi159.stdout:3325 write 0x975b0bf thru 0x975f8b9 (0x47fb bytes) 2024-09-17T22:58:02.977 INFO:teuthology.orchestra.run.smithi159.stdout:3326 trunc from 0xe319426 to 0x45621fd 2024-09-17T22:58:02.986 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:02.986 INFO:teuthology.orchestra.run.smithi159.stdout:3329 read 0xbb8aa1 thru 0xbc7613 (0xeb73 bytes) 2024-09-17T22:58:02.987 INFO:teuthology.orchestra.run.smithi159.stdout:3330 write 0x1abb90 thru 0x1b9564 (0xd9d5 bytes) 2024-09-17T22:58:02.991 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:02.991 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:02.992 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:03.001 INFO:teuthology.orchestra.run.smithi159.stdout:3332 trunc from 0x45621fd to 0x5037e17 2024-09-17T22:58:03.002 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:03.007 INFO:teuthology.orchestra.run.smithi159.stdout:3333 read 0x26f0aef thru 0x26fc878 (0xbd8a bytes) 2024-09-17T22:58:03.019 INFO:teuthology.orchestra.run.smithi159.stdout:3336 write 0x9d4ab7c thru 0x9d51e17 (0x729c bytes) 2024-09-17T22:58:03.024 INFO:teuthology.orchestra.run.smithi159.stdout:3337 read 0x15c17e thru 0x16311d (0x6fa0 bytes) 2024-09-17T22:58:03.024 INFO:teuthology.orchestra.run.smithi159.stdout:3341 write 0x66f2334 thru 0x66fca7d (0xa74a bytes) 2024-09-17T22:58:03.030 INFO:teuthology.orchestra.run.smithi159.stdout:3343 punch from 0x6b095d6 to 0x6b120b9, (0x8ae3 bytes) 2024-09-17T22:58:03.030 INFO:teuthology.orchestra.run.smithi159.stdout:3348 write 0x83f9572 thru 0x83f9ba4 (0x633 bytes) 2024-09-17T22:58:03.033 INFO:teuthology.orchestra.run.smithi159.stdout:3349 punch from 0x821aecc to 0x821da08, (0x2b3c bytes) 2024-09-17T22:58:03.033 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.033 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.033 INFO:teuthology.orchestra.run.smithi159.stdout:3352 punch from 0x9ce2189 to 0x9ce2250, (0xc7 bytes) 2024-09-17T22:58:03.039 INFO:teuthology.orchestra.run.smithi159.stdout:3353 write 0xb69806a thru 0xb6a47a2 (0xc739 bytes) 2024-09-17T22:58:03.043 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:03.041+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:03.045 INFO:teuthology.orchestra.run.smithi159.stdout:3355 read 0x6795676 thru 0x679c1eb (0x6b76 bytes) 2024-09-17T22:58:03.045 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.045 INFO:teuthology.orchestra.run.smithi159.stdout:3358 write 0xe06190 thru 0xe09d84 (0x3bf5 bytes) 2024-09-17T22:58:03.050 INFO:teuthology.orchestra.run.smithi159.stdout:3359 write 0x8827fd1 thru 0x882fc22 (0x7c52 bytes) 2024-09-17T22:58:03.054 INFO:teuthology.orchestra.run.smithi159.stdout:3364 read 0x5707d29 thru 0x571219c (0xa474 bytes) 2024-09-17T22:58:03.054 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.054 INFO:teuthology.orchestra.run.smithi159.stdout:3366 read 0x508b8ab thru 0x5093927 (0x807d bytes) 2024-09-17T22:58:03.054 INFO:teuthology.orchestra.run.smithi159.stdout:3368 punch from 0x8f82c6b to 0x8f886e5, (0x5a7a bytes) 2024-09-17T22:58:03.055 INFO:teuthology.orchestra.run.smithi159.stdout:3369 write 0x93ff338 thru 0x9403630 (0x42f9 bytes) 2024-09-17T22:58:03.068 INFO:teuthology.orchestra.run.smithi159.stdout:3372 trunc from 0xb6a47a3 to 0x2d3af00 2024-09-17T22:58:03.081 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.082 INFO:teuthology.orchestra.run.smithi159.stdout:3374 trunc from 0x2d3af00 to 0x3cc714a 2024-09-17T22:58:03.087 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.088 INFO:teuthology.orchestra.run.smithi159.stdout:3377 write 0x3178ac0 thru 0x3187d19 (0xf25a bytes) 2024-09-17T22:58:03.093 INFO:teuthology.orchestra.run.smithi159.stdout:3378 punch from 0x18a81c6 to 0x18af2ef, (0x7129 bytes) 2024-09-17T22:58:03.093 INFO:teuthology.orchestra.run.smithi159.stdout:3379 punch from 0xe595a2 to 0xe66a83, (0xd4e1 bytes) 2024-09-17T22:58:03.102 INFO:teuthology.orchestra.run.smithi159.stdout:3380 trunc from 0x3cc714a to 0xb6c4f89 2024-09-17T22:58:03.108 INFO:teuthology.orchestra.run.smithi159.stdout:3381 punch from 0x2015022 to 0x20245b2, (0xf590 bytes) 2024-09-17T22:58:03.108 INFO:teuthology.orchestra.run.smithi159.stdout:3385 writesame 0x1450ed3 thru 0x14561a2 data_size 0x64(0x52d0 bytes) 2024-09-17T22:58:03.115 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.115 INFO:teuthology.orchestra.run.smithi159.stdout:3387 writesame 0xb354fcc thru 0xb357dee data_size 0x3(0x2e23 bytes) 2024-09-17T22:58:03.118 INFO:teuthology.orchestra.run.smithi159.stdout:3388 write 0x4856549 thru 0x485c0f7 (0x5baf bytes) 2024-09-17T22:58:03.120 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:03.117+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:03.131 INFO:teuthology.orchestra.run.smithi159.stdout:3389 writesame 0xc41a33f thru 0xc42613d data_size 0x1f(0xbdff bytes) 2024-09-17T22:58:03.131 INFO:teuthology.orchestra.run.smithi159.stdout:3390 read 0x7744b6c thru 0x7748b4e (0x3fe3 bytes) 2024-09-17T22:58:03.133 INFO:teuthology.orchestra.run.smithi159.stdout:3391 trunc from 0xc42613e to 0xe74b706 2024-09-17T22:58:03.138 INFO:teuthology.orchestra.run.smithi159.stdout:3392 write 0x31e199f thru 0x31e2133 (0x795 bytes) 2024-09-17T22:58:03.140 INFO:teuthology.orchestra.run.smithi159.stdout:3395 write 0xbc303c6 thru 0xbc3f2e0 (0xef1b bytes) 2024-09-17T22:58:03.146 INFO:teuthology.orchestra.run.smithi159.stdout:3396 write 0xd9ce599 thru 0xd9d0a4a (0x24b2 bytes) 2024-09-17T22:58:03.149 INFO:teuthology.orchestra.run.smithi159.stdout:3397 read 0x8303a8e thru 0x8306774 (0x2ce7 bytes) 2024-09-17T22:58:03.150 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 59.290714661999999, 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:03.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:03.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:03.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:03.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.064031321999998, 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.162 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:58:03.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.166 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063996785999997, 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:58:03.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:03.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063942402999999, 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:58:03.173 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.174 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:58:03.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063917398999997, 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.179 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:58:03.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:58:03.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:58:03.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:58:03.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063896818000003, 2024-09-17T22:58:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:58:03.187 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.187 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:58:03.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063875535000001, 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.192 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:03.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:03.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:03.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063857009000003, 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:03.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063838836999999, 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.202 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:03.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063821009000002, 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:03.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:03.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.063505876000001, 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:03.211 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:03.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.062783605, 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:58:03.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.220 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.062739008999998, 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:58:03.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:58:03.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.225 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.062711686999997, 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:58:03.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:58:03.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.060892889000002, 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.060550083999999, 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.236 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:58:03.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.058278952000002, 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.240 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:58:03.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:58:03.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.058246857999997, 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:58:03.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:58:03.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:58:03.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.250 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.058220319999997, 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.251 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:58:03.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.058201427, 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:03.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:58:03.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:03.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:03.260 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:03.262 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:03.259+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:03.263 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3399 trunc from 0xe74b706 to 0xad98155 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3401 read 0x589e20b thru 0x58a9e07 (0xbbfd bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3403 write 0x59cc7e6 thru 0x59d662b (0x9e46 bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3404 writesame 0x2b195e9 thru 0x2b27550 data_size 0xc(0xdf68 bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3405 read 0x2342ffb thru 0x2345545 (0x254b bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3406 read 0x93aefcd thru 0x93b7bc3 (0x8bf7 bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3408 write 0x4070624 thru 0x4071edb (0x18b8 bytes) 2024-09-17T22:58:03.265 INFO:teuthology.orchestra.run.smithi159.stdout:3409 read 0x70896fb thru 0x708fb09 (0x640f bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3410 punch from 0x54409a9 to 0x544f6fd, (0xed54 bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3413 punch from 0x1615f27 to 0x1622bf4, (0xcccd bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3415 write 0x4aa1c02 thru 0x4ab15a7 (0xf9a6 bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3416 write 0x2cc723c thru 0x2cc89a9 (0x176e bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3417 punch from 0x66eb0e1 to 0x66f9e72, (0xed91 bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3419 write 0x24b41f3 thru 0x24c2fcf (0xeddd bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3420 trunc from 0xad98155 to 0x698f777 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3421 write 0x454d0e2 thru 0x45553b9 (0x82d8 bytes) 2024-09-17T22:58:03.266 INFO:teuthology.orchestra.run.smithi159.stdout:3423 read 0x16208ce thru 0x1622923 (0x2056 bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3424 read 0x38c3703 thru 0x38c93c2 (0x5cc0 bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3425 read 0x527a519 thru 0x5282344 (0x7e2c bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3426 trunc from 0x698f777 to 0x595ab3f 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3427 read 0x409c5b thru 0x41940a (0xf7b0 bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3428 punch from 0x583cd0f to 0x584605f, (0x9350 bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3429 trunc from 0x595ab3f to 0x1c91612 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3430 writesame 0x68ee4fb thru 0x68fcdff data_size 0x79(0xe905 bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3432 read 0x3ed26cc thru 0x3edf90a (0xd23f bytes) 2024-09-17T22:58:03.267 INFO:teuthology.orchestra.run.smithi159.stdout:3436 write 0xc8afc1e thru 0xc8b69ce (0x6db1 bytes) 2024-09-17T22:58:03.273 INFO:teuthology.orchestra.run.smithi159.stdout:3437 write 0x7801a38 thru 0x7809c11 (0x81da bytes) 2024-09-17T22:58:03.282 INFO:teuthology.orchestra.run.smithi159.stdout:3438 write 0x5a12c26 thru 0x5a20811 (0xdbec bytes) 2024-09-17T22:58:03.295 INFO:teuthology.orchestra.run.smithi159.stdout:3439 trunc from 0xc8b69cf to 0x96dd982 2024-09-17T22:58:03.314 INFO:teuthology.orchestra.run.smithi159.stdout:3440 write 0xbdc64ec thru 0xbdd30e0 (0xcbf5 bytes) 2024-09-17T22:58:03.319 INFO:teuthology.orchestra.run.smithi159.stdout:3441 punch from 0x8622255 to 0x862c6e4, (0xa48f bytes) 2024-09-17T22:58:03.327 INFO:teuthology.orchestra.run.smithi159.stdout:3442 write 0xe834012 thru 0xe837401 (0x33f0 bytes) 2024-09-17T22:58:03.330 INFO:teuthology.orchestra.run.smithi159.stdout:3443 writesame 0x13b99ef thru 0x13c4122 data_size 0xa4(0xa734 bytes) 2024-09-17T22:58:03.341 INFO:teuthology.orchestra.run.smithi159.stdout:3445 trunc from 0xe837402 to 0xb0f3aa3 2024-09-17T22:58:03.351 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.352 INFO:teuthology.orchestra.run.smithi159.stdout:3448 punch from 0xfd7d03 to 0xfdf7b2, (0x7aaf bytes) 2024-09-17T22:58:03.362 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:03.360+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:03.362 INFO:teuthology.orchestra.run.smithi159.stdout:3449 write 0xe8cb5c6 thru 0xe8d3055 (0x7a90 bytes) 2024-09-17T22:58:03.370 INFO:teuthology.orchestra.run.smithi159.stdout:3452 write 0x3ea1dae thru 0x3ea96ca (0x791d bytes) 2024-09-17T22:58:03.376 INFO:teuthology.orchestra.run.smithi159.stdout:3453 write 0x1750224 thru 0x1755d90 (0x5b6d bytes) 2024-09-17T22:58:03.377 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.387 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:03.387 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:03.387 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:03.387 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:03.392 INFO:teuthology.orchestra.run.smithi159.stdout:3455 trunc from 0xe8d3056 to 0x24be854 2024-09-17T22:58:03.401 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:03.416 INFO:teuthology.orchestra.run.smithi159.stdout:3457 trunc from 0x24be854 to 0xc662444 2024-09-17T22:58:03.427 INFO:teuthology.orchestra.run.smithi159.stdout:3458 trunc from 0xc662444 to 0x87c9428 2024-09-17T22:58:03.432 INFO:teuthology.orchestra.run.smithi159.stdout:3460 writesame 0x44f8616 thru 0x44fb8ad data_size 0x4(0x3298 bytes) 2024-09-17T22:58:03.448 INFO:teuthology.orchestra.run.smithi159.stdout:3461 write 0xe16ae5d thru 0xe16ff12 (0x50b6 bytes) 2024-09-17T22:58:03.451 INFO:teuthology.orchestra.run.smithi159.stdout:3462 writesame 0x90a68d3 thru 0x90af4d9 data_size 0x9(0x8c07 bytes) 2024-09-17T22:58:03.457 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.457 INFO:teuthology.orchestra.run.smithi159.stdout:3464 write 0xad0c46a thru 0xad1203a (0x5bd1 bytes) 2024-09-17T22:58:03.461 INFO:teuthology.orchestra.run.smithi159.stdout:3465 read 0xab710fb thru 0xab78a43 (0x7949 bytes) 2024-09-17T22:58:03.462 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.462 INFO:teuthology.orchestra.run.smithi159.stdout:3468 write 0x795d7d6 thru 0x795f4a3 (0x1cce bytes) 2024-09-17T22:58:03.463 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:03.460+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:03.465 INFO:teuthology.orchestra.run.smithi159.stdout:3469 punch from 0xc9a1979 to 0xc9a52c4, (0x394b bytes) 2024-09-17T22:58:03.465 INFO:teuthology.orchestra.run.smithi159.stdout:3471 punch from 0x521f3cf to 0x5222deb, (0x3a1c bytes) 2024-09-17T22:58:03.465 INFO:teuthology.orchestra.run.smithi159.stdout:3472 write 0xa87adbe thru 0xa881adc (0x6d1f bytes) 2024-09-17T22:58:03.479 INFO:teuthology.orchestra.run.smithi159.stdout:3473 trunc from 0xe16ff13 to 0x75ec07c 2024-09-17T22:58:03.500 INFO:teuthology.orchestra.run.smithi159.stdout:3474 writesame 0xb0c0ed1 thru 0xb0ce56a data_size 0xd(0xd69a bytes) 2024-09-17T22:58:03.500 INFO:teuthology.orchestra.run.smithi159.stdout:3476 read 0xafd6c5b thru 0xafdf1f6 (0x859c bytes) 2024-09-17T22:58:03.501 INFO:teuthology.orchestra.run.smithi159.stdout:3477 write 0x35d2ad3 thru 0x35d59e0 (0x2f0e bytes) 2024-09-17T22:58:03.504 INFO:teuthology.orchestra.run.smithi159.stdout:3478 read 0x69a9de9 thru 0x69aaea4 (0x10bc bytes) 2024-09-17T22:58:03.516 INFO:teuthology.orchestra.run.smithi159.stdout:3479 trunc from 0xb0ce56b to 0xbee3bf 2024-09-17T22:58:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:03.546 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:03.546 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:03.546 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:03.550 INFO:teuthology.orchestra.run.smithi159.stdout:3480 read 0x956496 thru 0x95d076 (0x6be1 bytes) 2024-09-17T22:58:03.556 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:03.560 INFO:teuthology.orchestra.run.smithi159.stdout:3481 trunc from 0xbee3bf to 0x854af22 2024-09-17T22:58:03.563 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:03.561+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:03.565 INFO:teuthology.orchestra.run.smithi159.stdout:3482 punch from 0x568632c to 0x568c255, (0x5f29 bytes) 2024-09-17T22:58:03.565 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.566 INFO:teuthology.orchestra.run.smithi159.stdout:3486 read 0x5c45ea8 thru 0x5c4f7ea (0x9943 bytes) 2024-09-17T22:58:03.569 INFO:teuthology.orchestra.run.smithi159.stdout:3487 trunc from 0x854af22 to 0xbbfb07f 2024-09-17T22:58:03.574 INFO:teuthology.orchestra.run.smithi159.stdout:3488 punch from 0xb7ed5eb to 0xb7ef3f3, (0x1e08 bytes) 2024-09-17T22:58:03.574 INFO:teuthology.orchestra.run.smithi159.stdout:3489 punch from 0xba810fa to 0xba8bb75, (0xaa7b bytes) 2024-09-17T22:58:03.576 INFO:teuthology.orchestra.run.smithi159.stdout:3490 trunc from 0xbbfb07f to 0xa73f5ad 2024-09-17T22:58:03.581 INFO:teuthology.orchestra.run.smithi159.stdout:3491 read 0xd28e31 thru 0xd367fd (0xd9cd bytes) 2024-09-17T22:58:03.581 INFO:teuthology.orchestra.run.smithi159.stdout:3492 read 0x81c7173 thru 0x81c9c3a (0x2ac8 bytes) 2024-09-17T22:58:03.582 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.582 INFO:teuthology.orchestra.run.smithi159.stdout:3494 read 0x9af0fb2 thru 0x9afb132 (0xa181 bytes) 2024-09-17T22:58:03.582 INFO:teuthology.orchestra.run.smithi159.stdout:3495 read 0x9f1d2d8 thru 0x9f20b15 (0x383e bytes) 2024-09-17T22:58:03.582 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.591 INFO:teuthology.orchestra.run.smithi159.stdout:3497 writesame 0xc499dac thru 0xc4a2f37 data_size 0x5c(0x918c bytes) 2024-09-17T22:58:03.591 INFO:teuthology.orchestra.run.smithi159.stdout:3498 punch from 0x15934ec to 0x1596b3b, (0x364f bytes) 2024-09-17T22:58:03.591 INFO:teuthology.orchestra.run.smithi159.stdout:3499 write 0xb23da7f thru 0xb23e207 (0x789 bytes) 2024-09-17T22:58:03.594 INFO:teuthology.orchestra.run.smithi159.stdout:3503 read 0xb7288e9 thru 0xb72d239 (0x4951 bytes) 2024-09-17T22:58:03.594 INFO:teuthology.orchestra.run.smithi159.stdout:3504 read 0x21f423b thru 0x21f5bc1 (0x1987 bytes) 2024-09-17T22:58:03.594 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.594 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.595 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.595 INFO:teuthology.orchestra.run.smithi159.stdout:3509 write 0xa17ad84 thru 0xa184af0 (0x9d6d bytes) 2024-09-17T22:58:03.604 INFO:teuthology.orchestra.run.smithi159.stdout:3510 write 0x3ab10f0 thru 0x3ab6d3e (0x5c4f bytes) 2024-09-17T22:58:03.608 INFO:teuthology.orchestra.run.smithi159.stdout:3512 read 0x94baa83 thru 0x94c771c (0xcc9a bytes) 2024-09-17T22:58:03.609 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.609 INFO:teuthology.orchestra.run.smithi159.stdout:3514 punch from 0x952bade to 0x953021b, (0x473d bytes) 2024-09-17T22:58:03.609 INFO:teuthology.orchestra.run.smithi159.stdout:3515 read 0x7687131 thru 0x76888ac (0x177c bytes) 2024-09-17T22:58:03.609 INFO:teuthology.orchestra.run.smithi159.stdout:3517 read 0x88c9caf thru 0x88d8c3c (0xef8e bytes) 2024-09-17T22:58:03.609 INFO:teuthology.orchestra.run.smithi159.stdout:3518 write 0x710a76c thru 0x71120c4 (0x7959 bytes) 2024-09-17T22:58:03.614 INFO:teuthology.orchestra.run.smithi159.stdout:3519 write 0xc01c79e thru 0xc01ff14 (0x3777 bytes) 2024-09-17T22:58:03.617 INFO:teuthology.orchestra.run.smithi159.stdout:3520 write 0x5d8e2f3 thru 0x5d979e3 (0x96f1 bytes) 2024-09-17T22:58:03.621 INFO:teuthology.orchestra.run.smithi159.stdout:3521 write 0x33925af thru 0x339aaac (0x84fe bytes) 2024-09-17T22:58:03.625 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.625 INFO:teuthology.orchestra.run.smithi159.stdout:3526 writesame 0x1376a4c thru 0x1379928 data_size 0x9(0x2edd bytes) 2024-09-17T22:58:03.629 INFO:teuthology.orchestra.run.smithi159.stdout:3528 write 0x194d22b thru 0x19564fa (0x92d0 bytes) 2024-09-17T22:58:03.645 INFO:teuthology.orchestra.run.smithi159.stdout:3529 write 0xcc3c62f thru 0xcc4a0bb (0xda8d bytes) 2024-09-17T22:58:03.654 INFO:teuthology.orchestra.run.smithi159.stdout:3530 writesame 0x7b3affd thru 0x7b3c119 data_size 0x1(0x111d bytes) 2024-09-17T22:58:03.661 INFO:teuthology.orchestra.run.smithi159.stdout:3531 writesame 0x1ed1430 thru 0x1ed388f data_size 0x30(0x2460 bytes) 2024-09-17T22:58:03.664 INFO:teuthology.orchestra.run.smithi159.stdout:3532 read 0xc22be63 thru 0xc237639 (0xb7d7 bytes) 2024-09-17T22:58:03.664 INFO:teuthology.orchestra.run.smithi159.stdout:3533 write 0x1d1f70b thru 0x1d2c28f (0xcb85 bytes) 2024-09-17T22:58:03.665 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:03.662+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:03.671 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.671 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.677 INFO:teuthology.orchestra.run.smithi159.stdout:3536 writesame 0xd9763cf thru 0xd97c943 data_size 0x1(0x6575 bytes) 2024-09-17T22:58:03.677 INFO:teuthology.orchestra.run.smithi159.stdout:3537 write 0x79f068e thru 0x79fedb3 (0xe726 bytes) 2024-09-17T22:58:03.681 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:03.681 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:03.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413455627000001, 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.682 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:58:03.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:58:03.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413351591000001, 2024-09-17T22:58:03.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:58:03.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:58:03.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.690 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413330330000001, 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.691 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:58:03.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.695 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413308987000001, 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:58:03.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413239529999998, 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.700 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.701 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:58:03.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413220551999999, 2024-09-17T22:58:03.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:03.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:03.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413191622999999, 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:58:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413165684999999, 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413137690999999, 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:58:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:58:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413116766999998, 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.722 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:58:03.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:58:03.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413070026, 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.726 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413047618, 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.730 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:58:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:03.734 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413024274000001, 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:58:03.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.413001585, 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:03.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.412979386, 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.743 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:58:03.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:58:03.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.412950599999999, 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:03.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.412927967000002, 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.751 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:58:03.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:58:03.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.412908714, 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.756 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:58:03.759 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.412881538000001, 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:03.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:03.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 19.235242542999998, 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:03.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:03.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:03.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:03.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:03.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:03.770 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:03.771 INFO:teuthology.orchestra.run.smithi159.stdout:3539 trunc from 0xd97c944 to 0x40ce697 2024-09-17T22:58:03.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:03.772 INFO:teuthology.orchestra.run.smithi159.stdout:3540 write 0xb9dac6c thru 0xb9e86d4 (0xda69 bytes) 2024-09-17T22:58:03.773 INFO:teuthology.orchestra.run.smithi159.stdout:3541 write 0x673a502 thru 0x67487cd (0xe2cc bytes) 2024-09-17T22:58:03.773 INFO:teuthology.orchestra.run.smithi159.stdout:3543 writesame 0x61c6cc0 thru 0x61cbe80 data_size 0x1(0x51c1 bytes) 2024-09-17T22:58:03.773 INFO:teuthology.orchestra.run.smithi159.stdout:3544 trunc from 0xb9e86d5 to 0x511d582 2024-09-17T22:58:03.773 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:03.769+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:03.779 INFO:teuthology.orchestra.run.smithi159.stdout:3545 writesame 0xa8c8ace thru 0xa8d736b data_size 0x4b(0xe89e bytes) 2024-09-17T22:58:03.779 INFO:teuthology.orchestra.run.smithi159.stdout:3546 write 0x7afcc87 thru 0x7b0a4e7 (0xd861 bytes) 2024-09-17T22:58:03.798 INFO:teuthology.orchestra.run.smithi159.stdout:3547 trunc from 0xa8d736c to 0x5924ad 2024-09-17T22:58:03.832 INFO:teuthology.orchestra.run.smithi159.stdout:3549 read 0xbe830 thru 0xc3932 (0x5103 bytes) 2024-09-17T22:58:03.857 INFO:teuthology.orchestra.run.smithi159.stdout:3550 write 0xe70a91f thru 0xe70c2b3 (0x1995 bytes) 2024-09-17T22:58:03.857 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:58:03.864 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.864 INFO:teuthology.orchestra.run.smithi159.stdout:3553 write 0x329bcbd thru 0x32a6cc6 (0xb00a bytes) 2024-09-17T22:58:03.871 INFO:teuthology.orchestra.run.smithi159.stdout:3555 read 0x7497c46 thru 0x7499fe8 (0x23a3 bytes) 2024-09-17T22:58:03.871 INFO:teuthology.orchestra.run.smithi159.stdout:3556 write 0xcf1e444 thru 0xcf26759 (0x8316 bytes) 2024-09-17T22:58:03.872 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:03.870+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:03.875 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.875 INFO:teuthology.orchestra.run.smithi159.stdout:3558 write 0xe826c2 thru 0xe8faca (0xd409 bytes) 2024-09-17T22:58:03.882 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.882 INFO:teuthology.orchestra.run.smithi159.stdout:3561 write 0x5417b44 thru 0x541cfad (0x546a bytes) 2024-09-17T22:58:03.889 INFO:teuthology.orchestra.run.smithi159.stdout:3562 writesame 0x49f0676 thru 0x49f6bb9 data_size 0x2(0x6544 bytes) 2024-09-17T22:58:03.893 INFO:teuthology.orchestra.run.smithi159.stdout:3566 writesame 0x40d51b4 thru 0x40df520 data_size 0x17(0xa36d bytes) 2024-09-17T22:58:03.898 INFO:teuthology.orchestra.run.smithi159.stdout:3567 read 0x35a8146 thru 0x35b42cf (0xc18a bytes) 2024-09-17T22:58:03.898 INFO:teuthology.orchestra.run.smithi159.stdout:3568 write 0x62fec45 thru 0x6307041 (0x83fd bytes) 2024-09-17T22:58:03.902 INFO:teuthology.orchestra.run.smithi159.stdout:3569 write 0x403bacc thru 0x404467f (0x8bb4 bytes) 2024-09-17T22:58:03.910 INFO:teuthology.orchestra.run.smithi159.stdout:3570 writesame 0x662a857 thru 0x663279a data_size 0x5a(0x7f44 bytes) 2024-09-17T22:58:03.911 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:03.911 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:03.911 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:03.911 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:03.919 INFO:teuthology.orchestra.run.smithi159.stdout:3572 read 0x909f422 thru 0x90ad3e3 (0xdfc2 bytes) 2024-09-17T22:58:03.919 INFO:teuthology.orchestra.run.smithi159.stdout:3573 write 0x70ea1ee thru 0x70f1fb8 (0x7dcb bytes) 2024-09-17T22:58:03.921 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:03.927 INFO:teuthology.orchestra.run.smithi159.stdout:3574 write 0x252e60e thru 0x25330d8 (0x4acb bytes) 2024-09-17T22:58:03.933 INFO:teuthology.orchestra.run.smithi159.stdout:3577 writesame 0x6357af4 thru 0x6362b45 data_size 0x3(0xb052 bytes) 2024-09-17T22:58:03.945 INFO:teuthology.orchestra.run.smithi159.stdout:3578 write 0x30c2a6d thru 0x30ccf40 (0xa4d4 bytes) 2024-09-17T22:58:03.951 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:03.952 INFO:teuthology.orchestra.run.smithi159.stdout:3580 read 0x5c0ab3b thru 0x5c12718 (0x7bde bytes) 2024-09-17T22:58:03.952 INFO:teuthology.orchestra.run.smithi159.stdout:3583 read 0xa0bcb6e thru 0xa0c2609 (0x5a9c bytes) 2024-09-17T22:58:03.952 INFO:teuthology.orchestra.run.smithi159.stdout:3584 writesame 0xa14f15f thru 0xa15a6d7 data_size 0x1(0xb579 bytes) 2024-09-17T22:58:03.968 INFO:teuthology.orchestra.run.smithi159.stdout:3586 writesame 0x148c7b1 thru 0x1497e9e data_size 0x69(0xb6ee bytes) 2024-09-17T22:58:03.973 INFO:teuthology.orchestra.run.smithi159.stdout:3587 write 0x21da6fb thru 0x21df9f1 (0x52f7 bytes) 2024-09-17T22:58:03.973 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:03.971+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:03.991 INFO:teuthology.orchestra.run.smithi159.stdout:3589 trunc from 0xe70c2b4 to 0x1e5b3ae 2024-09-17T22:58:04.009 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.022 INFO:teuthology.orchestra.run.smithi159.stdout:3591 write 0x8647bf9 thru 0x86480d5 (0x4dd bytes) 2024-09-17T22:58:04.026 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.037 INFO:teuthology.orchestra.run.smithi159.stdout:3594 writesame 0xd4c8603 thru 0xd4cb8f8 data_size 0xb(0x32f6 bytes) 2024-09-17T22:58:04.044 INFO:teuthology.orchestra.run.smithi159.stdout:3595 trunc from 0xd4cb8f9 to 0x79a3b71 2024-09-17T22:58:04.052 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:04.052 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:04.052 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:04.053 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:04.053 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:04.053 INFO:teuthology.orchestra.run.smithi159.stdout:3596 trunc from 0x79a3b71 to 0x6da75eb 2024-09-17T22:58:04.063 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:04.073 INFO:teuthology.orchestra.run.smithi159.stdout:3597 writesame 0xcf042fc thru 0xcf06116 data_size 0x7(0x1e1b bytes) 2024-09-17T22:58:04.074 INFO:teuthology.orchestra.run.smithi159.stdout:3598 write 0x149a0dd thru 0x14a03ea (0x630e bytes) 2024-09-17T22:58:04.074 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:04.071+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:04.079 INFO:teuthology.orchestra.run.smithi159.stdout:3599 write 0x4ce2195 thru 0x4ce6bcd (0x4a39 bytes) 2024-09-17T22:58:04.082 INFO:teuthology.orchestra.run.smithi159.stdout:3600 read 0xca121f thru 0xca77fd (0x65df bytes) 2024-09-17T22:58:04.083 INFO:teuthology.orchestra.run.smithi159.stdout:3601 punch from 0x2360399 to 0x2366fe8, (0x6c4f bytes) 2024-09-17T22:58:04.083 INFO:teuthology.orchestra.run.smithi159.stdout:3603 write 0x1187eea thru 0x1188ca6 (0xdbd bytes) 2024-09-17T22:58:04.086 INFO:teuthology.orchestra.run.smithi159.stdout:3604 read 0x230a9fa thru 0x23163e5 (0xb9ec bytes) 2024-09-17T22:58:04.087 INFO:teuthology.orchestra.run.smithi159.stdout:3605 read 0x4980e4e thru 0x4988f49 (0x80fc bytes) 2024-09-17T22:58:04.087 INFO:teuthology.orchestra.run.smithi159.stdout:3606 write 0xd680 thru 0xfccc (0x264d bytes) 2024-09-17T22:58:04.092 INFO:teuthology.orchestra.run.smithi159.stdout:3607 write 0xb14c7f thru 0xb157cb (0xb4d bytes) 2024-09-17T22:58:04.095 INFO:teuthology.orchestra.run.smithi159.stdout:3609 write 0xb38536a thru 0xb3930e2 (0xdd79 bytes) 2024-09-17T22:58:04.104 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.104 INFO:teuthology.orchestra.run.smithi159.stdout:3612 write 0x255df32 thru 0x256621c (0x82eb bytes) 2024-09-17T22:58:04.110 INFO:teuthology.orchestra.run.smithi159.stdout:3613 write 0xb888a8a thru 0xb88b1a7 (0x271e bytes) 2024-09-17T22:58:04.113 INFO:teuthology.orchestra.run.smithi159.stdout:3614 write 0x7a64396 thru 0x7a72c48 (0xe8b3 bytes) 2024-09-17T22:58:04.118 INFO:teuthology.orchestra.run.smithi159.stdout:3615 write 0xbfb81df thru 0xbfc49a9 (0xc7cb bytes) 2024-09-17T22:58:04.131 INFO:teuthology.orchestra.run.smithi159.stdout:3618 write 0xee11529 thru 0xee1398b (0x2463 bytes) 2024-09-17T22:58:04.134 INFO:teuthology.orchestra.run.smithi159.stdout:3619 write 0x5c7d54d thru 0x5c84d6e (0x7822 bytes) 2024-09-17T22:58:04.138 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.139 INFO:teuthology.orchestra.run.smithi159.stdout:3624 writesame 0x1a0df32 thru 0x1a12c22 data_size 0x1(0x4cf1 bytes) 2024-09-17T22:58:04.149 INFO:teuthology.orchestra.run.smithi159.stdout:3626 write 0xe1fc9a3 thru 0xe1fcd7c (0x3da bytes) 2024-09-17T22:58:04.152 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.153 INFO:teuthology.orchestra.run.smithi159.stdout:3632 write 0x13bd458 thru 0x13c2a50 (0x55f9 bytes) 2024-09-17T22:58:04.156 INFO:teuthology.orchestra.run.smithi159.stdout:3633 writesame 0xdc4d982 thru 0xdc51848 data_size 0xb(0x3ec7 bytes) 2024-09-17T22:58:04.160 INFO:teuthology.orchestra.run.smithi159.stdout:3634 write 0xe7fabf1 thru 0xe7fdbd6 (0x2fe6 bytes) 2024-09-17T22:58:04.167 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-17T22:58:04.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:04.167 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-17T22:58:04.167 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T22:58:04.167 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T22:58:04.168 INFO:teuthology.orchestra.run.smithi159.stdout:3635 read 0x7c63882 thru 0x7c67f36 (0x46b5 bytes) 2024-09-17T22:58:04.168 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.168 INFO:teuthology.orchestra.run.smithi159.stdout:3638 writesame 0x81c97b thru 0x822996 data_size 0x2(0x601c bytes) 2024-09-17T22:58:04.170 INFO:tasks.ceph.osd.5:Started 2024-09-17T22:58:04.171 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:04.176 INFO:teuthology.orchestra.run.smithi159.stdout:3639 write 0x619a423 thru 0x619e63e (0x421c bytes) 2024-09-17T22:58:04.183 INFO:teuthology.orchestra.run.smithi159.stdout:3640 writesame 0x6908751 thru 0x6912ceb data_size 0x5(0xa59b bytes) 2024-09-17T22:58:04.191 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16134 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.933074988000001, 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036277120000000003, 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16134 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255129+0000", 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255131+0000", 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.916e-06 2024-09-17T22:58:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255137+0000", 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3509999999999998e-06 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255142+0000", 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.882e-06 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291253+0000", 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036111153 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291337+0000", 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3535000000000006e-05 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291405+0000", 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8858999999999999e-05 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16135 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.933055015000001, 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036429299999999998, 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16135 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255149+0000", 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255151+0000", 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255157+0000", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255162+0000", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291437+0000", 2024-09-17T22:58:04.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036274726 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291514+0000", 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7342000000000006e-05 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291578+0000", 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3634999999999996e-05 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.201 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16159 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.933035275999998, 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036685328000000003, 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16159 2024-09-17T22:58:04.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255170+0000", 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255177+0000", 2024-09-17T22:58:04.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4989999999999999e-06 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255193+0000", 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909999999999998e-05 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291665+0000", 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036471816999999997 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291751+0000", 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6509999999999997e-05 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291853+0000", 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102099 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16161 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.206 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932997725, 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036865504, 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16161 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.207 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255208+0000", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3250000000000002e-06 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255218+0000", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4080000000000004e-06 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255223+0000", 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:58:04.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291884+0000", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036661457000000001 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291966+0000", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1920000000000002e-05 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292071+0000", 2024-09-17T22:58:04.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000104608 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16163 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932973274999998, 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037090170999999998, 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.211 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16163 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.212 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255233+0000", 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255242+0000", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0750000000000004e-06 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255247+0000", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292101+0000", 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036854162000000003 2024-09-17T22:58:04.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292174+0000", 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2805000000000005e-05 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292320+0000", 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014646799999999999 2024-09-17T22:58:04.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16165 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1728512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932949836999999, 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037305086000000001, 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.216 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16165 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:04.217 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255254+0000", 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255256+0000", 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.12e-06 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255265+0000", 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8959999999999992e-06 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255270+0000", 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8760000000000004e-06 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292365+0000", 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037095377999999998 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292452+0000", 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6947000000000006e-05 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292558+0000", 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010643 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16167 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932926801000001, 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037507133999999998, 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16167 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.221 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255277+0000", 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255279+0000", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255288+0000", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8429999999999999e-06 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255293+0000", 2024-09-17T22:58:04.223 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292603+0000", 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037309869000000002 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292681+0000", 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8484000000000003e-05 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.224 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292784+0000", 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010224 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16169 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.93290378, 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037775437000000002, 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16169 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.226 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255300+0000", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255302+0000", 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255308+0000", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255313+0000", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292863+0000", 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037549224999999999 2024-09-17T22:58:04.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292969+0000", 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010604500000000001 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293075+0000", 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010620699999999999 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16171 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1875968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932883885999999, 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037958219000000001, 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16171 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255320+0000", 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2899999999999999e-07 2024-09-17T22:58:04.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255322+0000", 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.906e-06 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255337+0000", 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5452999999999999e-05 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.232 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255343+0000", 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293106+0000", 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037763707000000001 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293186+0000", 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0153000000000003e-05 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.233 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293278+0000", 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1114999999999998e-05 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16172 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1925120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932853603000002, 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038145196999999999, 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.234 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16172 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8400000000000004e-07 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255359+0000", 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:04.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255367+0000", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0649999999999999e-06 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255372+0000", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1390000000000003e-06 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293318+0000", 2024-09-17T22:58:04.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037945566 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293399+0000", 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1168999999999995e-05 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293495+0000", 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6162000000000005e-05 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16173 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932822408, 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038380816999999998, 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16173 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.239 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255388+0000", 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255402+0000", 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4532000000000001e-05 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255408+0000", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7740000000000001e-06 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293577+0000", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038169526000000002 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293664+0000", 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6312999999999998e-05 2024-09-17T22:58:04.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293762+0000", 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7947000000000003e-05 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16174 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932787652000002, 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038531986999999997, 2024-09-17T22:58:04.243 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16174 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255418+0000", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255426+0000", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1620000000000002e-06 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255436+0000", 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0851e-05 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293804+0000", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038367215000000003 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293886+0000", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2189999999999997e-05 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293948+0000", 2024-09-17T22:58:04.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1709000000000002e-05 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16176 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.932759699000002, 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038738529000000001, 2024-09-17T22:58:04.248 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16176 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255446+0000", 2024-09-17T22:58:04.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255453+0000", 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8170000000000001e-06 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255458+0000", 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293991+0000", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038532376 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294084+0000", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3614000000000002e-05 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294182+0000", 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7949999999999996e-05 2024-09-17T22:58:04.252 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16239 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.929387240000001, 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036338338999999997, 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.253 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16239 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258819+0000", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3e-06 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258825+0000", 2024-09-17T22:58:04.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258830+0000", 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294968+0000", 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036137423000000002 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295061+0000", 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3177999999999995e-05 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295154+0000", 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3645000000000002e-05 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16241 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1835008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.929362674, 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036566098999999998, 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16241 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4770000000000002e-06 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258850+0000", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5490000000000003e-06 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258855+0000", 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.904e-06 2024-09-17T22:58:04.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295247+0000", 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036392378000000003 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295334+0000", 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7058999999999994e-05 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295407+0000", 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2287999999999996e-05 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16243 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.929340808999999, 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036705072999999998, 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.262 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16243 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258863+0000", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258866+0000", 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.678e-06 2024-09-17T22:58:04.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258872+0000", 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258877+0000", 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295445+0000", 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036567582000000001 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295506+0000", 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1895000000000003e-05 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295568+0000", 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1085000000000002e-05 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.266 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16245 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1933312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.929014222999999, 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036671423000000002, 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.267 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16245 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259190+0000", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259194+0000", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259205+0000", 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0468e-05 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259212+0000", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1380000000000001e-06 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295666+0000", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036453700999999998 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295753+0000", 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7321e-05 2024-09-17T22:58:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295860+0000", 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010761599999999999 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16246 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.928979666, 2024-09-17T22:58:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036882379, 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16246 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6599999999999996e-07 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259227+0000", 2024-09-17T22:58:04.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259234+0000", 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8539999999999999e-06 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259239+0000", 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295903+0000", 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036664018999999999 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295993+0000", 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9944000000000005e-05 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296106+0000", 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113158 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.275 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16248 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.928957495999999, 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037070651000000003, 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16248 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259248+0000", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259254+0000", 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:58:04.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259260+0000", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4489999999999998e-06 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296156+0000", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036896089999999999 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296238+0000", 2024-09-17T22:58:04.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2616999999999999e-05 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296316+0000", 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7964000000000001e-05 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16249 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 27.928786607999999, 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037100148999999999, 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16249 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8700000000000005e-07 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259421+0000", 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0470000000000004e-06 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259431+0000", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4190000000000008e-06 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259437+0000", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4520000000000002e-06 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296352+0000", 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036914354000000003 2024-09-17T22:58:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296427+0000", 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5603000000000001e-05 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296517+0000", 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9586999999999995e-05 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:04.286 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:04.286 INFO:teuthology.orchestra.run.smithi159.stdout:3642 read 0x78f93e7 thru 0x7908102 (0xed1c bytes) 2024-09-17T22:58:04.286 INFO:teuthology.orchestra.run.smithi159.stdout:3643 punch from 0x46c5726 to 0x46ca666, (0x4f40 bytes) 2024-09-17T22:58:04.287 INFO:teuthology.orchestra.run.smithi159.stdout:3645 write 0x3df5bd5 thru 0x3dfd8f6 (0x7d22 bytes) 2024-09-17T22:58:04.287 INFO:teuthology.orchestra.run.smithi159.stdout:3646 read 0xd3f85df thru 0xd3f8ea1 (0x8c3 bytes) 2024-09-17T22:58:04.287 INFO:teuthology.orchestra.run.smithi159.stdout:3648 read 0xba54a39 thru 0xba5e5ee (0x9bb6 bytes) 2024-09-17T22:58:04.288 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:04.290 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:04.212+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:04.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3649 trunc from 0xee1398c to 0x837a8d5 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3650 punch from 0x25aab96 to 0x25acd1b, (0x2185 bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3653 write 0x7a773c7 thru 0x7a7e0b9 (0x6cf3 bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3654 writesame 0x9906a4b thru 0x990f10e data_size 0x73(0x86c4 bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3658 write 0x5d29ad thru 0x5db6fc (0x8d50 bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3659 write 0x65906d6 thru 0x659b21c (0xab47 bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3660 read 0x25fb984 thru 0x260885f (0xcedc bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3663 write 0xa64c19a thru 0xa64c8a3 (0x70a bytes) 2024-09-17T22:58:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:3664 trunc from 0xa64c8a4 to 0xbf106f1 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi159.stdout:3665 trunc from 0xbf106f1 to 0x667ba70 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi159.stdout:3666 write 0x8e00694 thru 0x8e08b3d (0x84aa bytes) 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi159.stdout:3667 read 0x6434b3c thru 0x6435bc5 (0x108a bytes) 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi159.stdout:3668 write 0xa515078 thru 0xa5171ae (0x2137 bytes) 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-17T22:58:04.292 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:config diff 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:config diff get 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:config get 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:config help [] 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:config set ... 2024-09-17T22:58:04.293 INFO:teuthology.orchestra.run.smithi171.stderr:admin_socket: invalid command 2024-09-17T22:58:04.293 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-17T22:58:04.300 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:04.302 INFO:teuthology.orchestra.run.smithi159.stdout:3670 trunc from 0xa5171af to 0x1fe91c6 2024-09-17T22:58:04.303 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:04.298+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:04.314 INFO:teuthology.orchestra.run.smithi159.stdout:3671 read 0x1aec4c1 thru 0x1af1018 (0x4b58 bytes) 2024-09-17T22:58:04.336 INFO:teuthology.orchestra.run.smithi159.stdout:3672 write 0xda1fa0f thru 0xda299ee (0x9fe0 bytes) 2024-09-17T22:58:04.346 INFO:teuthology.orchestra.run.smithi159.stdout:3673 write 0x1f6dbfe thru 0x1f79754 (0xbb57 bytes) 2024-09-17T22:58:04.358 INFO:teuthology.orchestra.run.smithi159.stdout:3674 write 0x6aefdcb thru 0x6af1fb8 (0x21ee bytes) 2024-09-17T22:58:04.361 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.361 INFO:teuthology.orchestra.run.smithi159.stdout:3676 read 0xe8c721 thru 0xe8fca0 (0x3580 bytes) 2024-09-17T22:58:04.362 INFO:teuthology.orchestra.run.smithi159.stdout:3677 read 0x76051a4 thru 0x7609e79 (0x4cd6 bytes) 2024-09-17T22:58:04.362 INFO:teuthology.orchestra.run.smithi159.stdout:3678 writesame 0xd862dca thru 0xd86f691 data_size 0x64(0xc8c8 bytes) 2024-09-17T22:58:04.366 INFO:teuthology.orchestra.run.smithi159.stdout:3679 read 0xa1ce3b7 thru 0xa1d46ee (0x6338 bytes) 2024-09-17T22:58:04.367 INFO:teuthology.orchestra.run.smithi159.stdout:3680 trunc from 0xda299ef to 0xd5ee36a 2024-09-17T22:58:04.378 INFO:teuthology.orchestra.run.smithi159.stdout:3681 write 0xc2d16e3 thru 0xc2d259a (0xeb8 bytes) 2024-09-17T22:58:04.381 INFO:teuthology.orchestra.run.smithi159.stdout:3682 read 0xcafb0cf thru 0xcafb128 (0x5a bytes) 2024-09-17T22:58:04.381 INFO:teuthology.orchestra.run.smithi159.stdout:3683 punch from 0x668de2a to 0x66969a3, (0x8b79 bytes) 2024-09-17T22:58:04.381 INFO:teuthology.orchestra.run.smithi159.stdout:3684 writesame 0xadd077d thru 0xadd17b5 data_size 0x1(0x1039 bytes) 2024-09-17T22:58:04.390 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:04.388+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:04.398 INFO:teuthology.orchestra.run.smithi159.stdout:3686 trunc from 0xd5ee36a to 0x2910cff 2024-09-17T22:58:04.421 INFO:teuthology.orchestra.run.smithi159.stdout:3687 trunc from 0x2910cff to 0xd615843 2024-09-17T22:58:04.429 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:04.429 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:04.429 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:04.429 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:04.438 INFO:teuthology.orchestra.run.smithi159.stdout:3688 trunc from 0xd615843 to 0x4aff4ef 2024-09-17T22:58:04.444 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:04.444 INFO:teuthology.orchestra.run.smithi159.stdout:3691 write 0x210ee81 thru 0x211bec7 (0xd047 bytes) 2024-09-17T22:58:04.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:04.447 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:04.460 INFO:teuthology.orchestra.run.smithi159.stdout:3692 writesame 0x51cbb70 thru 0x51d681d data_size 0x2e(0xacae bytes) 2024-09-17T22:58:04.460 INFO:teuthology.orchestra.run.smithi159.stdout:3693 read 0x1d612d5 thru 0x1d6f0a4 (0xddd0 bytes) 2024-09-17T22:58:04.466 INFO:teuthology.orchestra.run.smithi159.stdout:3695 trunc from 0x51d681e to 0x9bf9e77 2024-09-17T22:58:04.467 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:04.467 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:04.473 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.477 INFO:teuthology.orchestra.run.smithi159.stdout:3697 trunc from 0x9bf9e77 to 0x60a3402 2024-09-17T22:58:04.487 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:04.491 INFO:teuthology.orchestra.run.smithi159.stdout:3699 write 0x6ff23d9 thru 0x6fff795 (0xd3bd bytes) 2024-09-17T22:58:04.491 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:04.489+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:04.509 INFO:teuthology.orchestra.run.smithi159.stdout:3700 write 0xc23cdf4 thru 0xc23e963 (0x1b70 bytes) 2024-09-17T22:58:04.509 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:04.509 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:04.519 INFO:teuthology.orchestra.run.smithi159.stdout:3701 writesame 0xdf4d6e5 thru 0xdf4ebcc data_size 0xc(0x14e8 bytes) 2024-09-17T22:58:04.519 INFO:teuthology.orchestra.run.smithi159.stdout:3702 punch from 0x860e636 to 0x861b577, (0xcf41 bytes) 2024-09-17T22:58:04.519 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.519 INFO:teuthology.orchestra.run.smithi159.stdout:3705 write 0x327a7df thru 0x327bbc0 (0x13e2 bytes) 2024-09-17T22:58:04.521 INFO:teuthology.orchestra.run.smithi159.stdout:3706 punch from 0xae0d7a3 to 0xae18e7d, (0xb6da bytes) 2024-09-17T22:58:04.521 INFO:teuthology.orchestra.run.smithi159.stdout:3708 read 0x9ac3ec1 thru 0x9ac7c4c (0x3d8c bytes) 2024-09-17T22:58:04.522 INFO:teuthology.orchestra.run.smithi159.stdout:3709 write 0xdbfb24 thru 0xdcad0c (0xb1e9 bytes) 2024-09-17T22:58:04.528 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:04.546 INFO:teuthology.orchestra.run.smithi159.stdout:3711 trunc from 0xdf4ebcd to 0x2838d4 2024-09-17T22:58:04.563 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:04.563 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:04.579 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:04.579 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:04.585 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:04.585 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:04.590 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:04.590 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:04.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:04.592 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:04.589+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:04.598 INFO:teuthology.orchestra.run.smithi159.stdout:3712 writesame 0x8eaccd8 thru 0x8eb7f69 data_size 0x39(0xb292 bytes) 2024-09-17T22:58:04.606 INFO:teuthology.orchestra.run.smithi159.stdout:3713 write 0xa3e16fc thru 0xa3efcb5 (0xe5ba bytes) 2024-09-17T22:58:04.614 INFO:teuthology.orchestra.run.smithi159.stdout:3714 write 0x41c0b3f thru 0x41d0a5c (0xff1e bytes) 2024-09-17T22:58:04.620 INFO:teuthology.orchestra.run.smithi159.stdout:3715 read 0x5519166 thru 0x551d1a9 (0x4044 bytes) 2024-09-17T22:58:04.621 INFO:teuthology.orchestra.run.smithi159.stdout:3716 writesame 0xc22ab3 thru 0xc304e1 data_size 0x1(0xda2f bytes) 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:3720 read 0x8082bc1 thru 0x808b0e7 (0x8527 bytes) 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:3724 punch from 0x1ac9d6a to 0x1ad5cc4, (0xbf5a bytes) 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:3725 read 0x93e4621 thru 0x93e9241 (0x4c21 bytes) 2024-09-17T22:58:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:3726 write 0x96d05a5 thru 0x96d53cc (0x4e28 bytes) 2024-09-17T22:58:04.642 INFO:teuthology.orchestra.run.smithi159.stdout:3727 punch from 0x2375adc to 0x2379148, (0x366c bytes) 2024-09-17T22:58:04.642 INFO:teuthology.orchestra.run.smithi159.stdout:3728 punch from 0x6b3e29b to 0x6b3f3c9, (0x112e bytes) 2024-09-17T22:58:04.642 INFO:teuthology.orchestra.run.smithi159.stdout:3729 read 0x75b7db2 thru 0x75c303c (0xb28b bytes) 2024-09-17T22:58:04.644 INFO:teuthology.orchestra.run.smithi159.stdout:3731 trunc from 0xa3efcb6 to 0x8a7507c 2024-09-17T22:58:04.653 INFO:teuthology.orchestra.run.smithi159.stdout:3733 punch from 0x3d91836 to 0x3d97eee, (0x66b8 bytes) 2024-09-17T22:58:04.661 INFO:teuthology.orchestra.run.smithi159.stdout:3737 trunc from 0x8a7507c to 0x28ccb08 2024-09-17T22:58:04.672 INFO:teuthology.orchestra.run.smithi159.stdout:3738 read 0xb64868 thru 0xb746e3 (0xfe7c bytes) 2024-09-17T22:58:04.679 INFO:teuthology.orchestra.run.smithi159.stdout:3739 write 0x2cceee7 thru 0x2cdcaa4 (0xdbbe bytes) 2024-09-17T22:58:04.687 INFO:teuthology.orchestra.run.smithi159.stdout:3740 writesame 0x24103f8 thru 0x241b1cd data_size 0x3(0xadd6 bytes) 2024-09-17T22:58:04.693 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:04.690+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:04.694 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.694 INFO:teuthology.orchestra.run.smithi159.stdout:3744 punch from 0xe63d19 to 0xe64b1a, (0xe01 bytes) 2024-09-17T22:58:04.698 INFO:teuthology.orchestra.run.smithi159.stdout:3746 trunc from 0x2cdcaa5 to 0x5a514b5 2024-09-17T22:58:04.703 INFO:teuthology.orchestra.run.smithi159.stdout:3747 read 0x5a822 thru 0x63120 (0x88ff bytes) 2024-09-17T22:58:04.704 INFO:teuthology.orchestra.run.smithi159.stdout:3748 read 0x22a8cb4 thru 0x22b275c (0x9aa9 bytes) 2024-09-17T22:58:04.704 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.704 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.705 INFO:teuthology.orchestra.run.smithi159.stdout:3751 writesame 0x5664e9 thru 0x574f6c data_size 0x6(0xea84 bytes) 2024-09-17T22:58:04.712 INFO:teuthology.orchestra.run.smithi159.stdout:3752 write 0x1179bff thru 0x1183e0b (0xa20d bytes) 2024-09-17T22:58:04.717 INFO:teuthology.orchestra.run.smithi159.stdout:3753 punch from 0x4913fd2 to 0x491c115, (0x8143 bytes) 2024-09-17T22:58:04.717 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.717 INFO:teuthology.orchestra.run.smithi159.stdout:3755 read 0x44ab34d thru 0x44b91c5 (0xde79 bytes) 2024-09-17T22:58:04.718 INFO:teuthology.orchestra.run.smithi159.stdout:3758 punch from 0x1947e24 to 0x194f453, (0x762f bytes) 2024-09-17T22:58:04.718 INFO:teuthology.orchestra.run.smithi159.stdout:3763 write 0x28b4701 thru 0x28b8e6c (0x476c bytes) 2024-09-17T22:58:04.721 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:04.721 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:04.736 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:04.736 INFO:teuthology.orchestra.run.smithi159.stdout:3764 write 0xe220d4e thru 0xe22cb31 (0xbde4 bytes) 2024-09-17T22:58:04.742 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.742 INFO:teuthology.orchestra.run.smithi159.stdout:3769 trunc from 0xe22cb32 to 0xe9df38a 2024-09-17T22:58:04.747 INFO:teuthology.orchestra.run.smithi159.stdout:3772 trunc from 0xe9df38a to 0xdb1b0cf 2024-09-17T22:58:04.757 INFO:teuthology.orchestra.run.smithi159.stdout:3773 write 0x51a269d thru 0x51a29dc (0x340 bytes) 2024-09-17T22:58:04.760 INFO:teuthology.orchestra.run.smithi159.stdout:3774 read 0xd914ad9 thru 0xd92414c (0xf674 bytes) 2024-09-17T22:58:04.766 INFO:teuthology.orchestra.run.smithi159.stdout:3775 trunc from 0xdb1b0cf to 0x8f57b8d 2024-09-17T22:58:04.772 INFO:teuthology.orchestra.run.smithi159.stdout:3776 read 0x6fdc633 thru 0x6fe3485 (0x6e53 bytes) 2024-09-17T22:58:04.784 INFO:teuthology.orchestra.run.smithi159.stdout:3777 writesame 0xcfa3bda thru 0xcfb3a85 data_size 0x12(0xfeac bytes) 2024-09-17T22:58:04.785 INFO:teuthology.orchestra.run.smithi159.stdout:3778 trunc from 0xcfb3a86 to 0xbfcc8b1 2024-09-17T22:58:04.791 INFO:teuthology.orchestra.run.smithi159.stdout:3779 read 0x20391e7 thru 0x203a844 (0x165e bytes) 2024-09-17T22:58:04.791 INFO:teuthology.orchestra.run.smithi159.stdout:3781 read 0xb80464 thru 0xb873f3 (0x6f90 bytes) 2024-09-17T22:58:04.794 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:04.791+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:3782 trunc from 0xbfcc8b1 to 0x63156b4 2024-09-17T22:58:04.805 INFO:teuthology.orchestra.run.smithi159.stdout:3783 write 0x3ebb061 thru 0x3ec7ecf (0xce6f bytes) 2024-09-17T22:58:04.811 INFO:teuthology.orchestra.run.smithi159.stdout:3784 read 0xf7b9ad thru 0xf7df73 (0x25c7 bytes) 2024-09-17T22:58:04.812 INFO:teuthology.orchestra.run.smithi159.stdout:3785 read 0x50d924 thru 0x515ae9 (0x81c6 bytes) 2024-09-17T22:58:04.812 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.823 INFO:teuthology.orchestra.run.smithi159.stdout:3787 write 0xbce2269 thru 0xbce8758 (0x64f0 bytes) 2024-09-17T22:58:04.841 INFO:teuthology.orchestra.run.smithi159.stdout:3789 trunc from 0xbce8759 to 0xac26f5 2024-09-17T22:58:04.855 INFO:teuthology.orchestra.run.smithi159.stdout:3790 read 0xa34de0 thru 0xa406af (0xb8d0 bytes) 2024-09-17T22:58:04.855 INFO:teuthology.orchestra.run.smithi159.stdout:3791 punch from 0x111de9 to 0x115838, (0x3a4f bytes) 2024-09-17T22:58:04.855 INFO:teuthology.orchestra.run.smithi159.stdout:3792 read 0x51bc9c thru 0x523f45 (0x82aa bytes) 2024-09-17T22:58:04.869 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:04.870 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:04.875 INFO:teuthology.orchestra.run.smithi159.stdout:3793 write 0xc991194 thru 0xc992c54 (0x1ac1 bytes) 2024-09-17T22:58:04.878 INFO:teuthology.orchestra.run.smithi159.stdout:3794 writesame 0x6705b4a thru 0x6706f6f data_size 0x1(0x1426 bytes) 2024-09-17T22:58:04.880 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:04.894 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:04.892+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:04.895 INFO:teuthology.orchestra.run.smithi159.stdout:3796 trunc from 0xc992c55 to 0x24c3b25 2024-09-17T22:58:04.914 INFO:teuthology.orchestra.run.smithi159.stdout:3797 write 0x53846a3 thru 0x5385186 (0xae4 bytes) 2024-09-17T22:58:04.917 INFO:teuthology.orchestra.run.smithi159.stdout:3798 read 0x1d5c19 thru 0x1e1cd2 (0xc0ba bytes) 2024-09-17T22:58:04.918 INFO:teuthology.orchestra.run.smithi159.stdout:3800 read 0x537ce23 thru 0x5385186 (0x8364 bytes) 2024-09-17T22:58:04.935 INFO:teuthology.orchestra.run.smithi159.stdout:3802 writesame 0xec18592 thru 0xec26c2d data_size 0x2(0xe69c bytes) 2024-09-17T22:58:04.935 INFO:teuthology.orchestra.run.smithi159.stdout:3803 writesame 0x564bcb7 thru 0x5651997 data_size 0x1f(0x5ce1 bytes) 2024-09-17T22:58:04.940 INFO:teuthology.orchestra.run.smithi159.stdout:3804 punch from 0x3439476 to 0x343e2a7, (0x4e31 bytes) 2024-09-17T22:58:04.955 INFO:teuthology.orchestra.run.smithi159.stdout:3805 trunc from 0xec26c2e to 0x1f30b63 2024-09-17T22:58:04.965 INFO:teuthology.orchestra.run.smithi159.stdout:3806 punch from 0x17abd85 to 0x17b180b, (0x5a86 bytes) 2024-09-17T22:58:04.965 INFO:teuthology.orchestra.run.smithi159.stdout:3807 read 0x1b5c213 thru 0x1b6a884 (0xe672 bytes) 2024-09-17T22:58:04.979 INFO:teuthology.orchestra.run.smithi159.stdout:3808 write 0x962fbea thru 0x9633781 (0x3b98 bytes) 2024-09-17T22:58:04.983 INFO:teuthology.orchestra.run.smithi159.stdout:3810 punch from 0x2635d29 to 0x2644a91, (0xed68 bytes) 2024-09-17T22:58:04.983 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.983 INFO:teuthology.orchestra.run.smithi159.stdout:3814 read 0x19fbbef thru 0x1a03456 (0x7868 bytes) 2024-09-17T22:58:04.983 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:04.984 INFO:teuthology.orchestra.run.smithi159.stdout:3816 trunc from 0x9633782 to 0x8b06bbe 2024-09-17T22:58:04.993 INFO:teuthology.orchestra.run.smithi159.stdout:3817 read 0x1aa055 thru 0x1b867c (0xe628 bytes) 2024-09-17T22:58:04.994 INFO:teuthology.orchestra.run.smithi159.stdout:3819 writesame 0x5de0a0d thru 0x5def88f data_size 0x1(0xee83 bytes) 2024-09-17T22:58:04.995 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:04.992+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:05.010 INFO:teuthology.orchestra.run.smithi159.stdout:3821 write 0x80a57ff thru 0x80afda8 (0xa5aa bytes) 2024-09-17T22:58:05.018 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.019 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.028 INFO:teuthology.orchestra.run.smithi159.stdout:3822 write 0xe460a31 thru 0xe46ef88 (0xe558 bytes) 2024-09-17T22:58:05.031 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:05.034 INFO:teuthology.orchestra.run.smithi159.stdout:3823 punch from 0x4f75ddb to 0x4f8187a, (0xba9f bytes) 2024-09-17T22:58:05.034 INFO:teuthology.orchestra.run.smithi159.stdout:3824 punch from 0xcb54b8d to 0xcb56e09, (0x227c bytes) 2024-09-17T22:58:05.034 INFO:teuthology.orchestra.run.smithi159.stdout:3825 clone 26 order 23 su 4194304 sc 12 2024-09-17T22:58:05.096 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:05.094+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:05.136 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.137 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.145 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:05.197 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:05.195+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:05.249 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.249 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.257 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:05.302 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:05.300+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:05.361 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.361 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:05.398 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:05.397+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:05.499 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:05.497+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:05.513 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.514 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.522 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:05.600 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:05.598+0000 7fbef5989640 -1 Falling back to public interface 2024-09-17T22:58:05.601 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:05.599+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:05.666 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.667 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:05.701 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:05.700+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:05.802 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:05.800+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:05.819 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.819 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.827 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:05.877 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:58:05.903 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:05.900+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:05.938 INFO:teuthology.orchestra.run.smithi159.stdout:flattening image image_client.0-clone25 2024-09-17T22:58:05.938 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:05.938 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:05.948 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:06.032 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:06.030+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:06.092 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:06.093 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:06.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:06.115 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T22:58:06.143 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:06.141+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:06.163 INFO:teuthology.orchestra.run.smithi159.stdout:3826 write 0x1a8913f thru 0x1a8cdae (0x3c70 bytes) 2024-09-17T22:58:06.170 INFO:teuthology.orchestra.run.smithi159.stdout:3829 punch from 0x1eea0b9 to 0x1ef23f0, (0x8337 bytes) 2024-09-17T22:58:06.171 INFO:teuthology.orchestra.run.smithi159.stdout:3830 read 0x3010cdd thru 0x302037f (0xf6a3 bytes) 2024-09-17T22:58:06.176 INFO:teuthology.orchestra.run.smithi159.stdout:3831 writesame 0xe4b4f42 thru 0xe4bf8b2 data_size 0x13(0xa971 bytes) 2024-09-17T22:58:06.179 INFO:teuthology.orchestra.run.smithi159.stdout:3832 write 0xc9b81f3 thru 0xc9c2044 (0x9e52 bytes) 2024-09-17T22:58:06.183 INFO:teuthology.orchestra.run.smithi159.stdout:3833 writesame 0x6513f24 thru 0x6515dea data_size 0x1(0x1ec7 bytes) 2024-09-17T22:58:06.189 INFO:teuthology.orchestra.run.smithi159.stdout:3834 write 0xcc7d249 thru 0xcc86c88 (0x9a40 bytes) 2024-09-17T22:58:06.192 INFO:teuthology.orchestra.run.smithi159.stdout:3835 write 0x6351f29 thru 0x6355737 (0x380f bytes) 2024-09-17T22:58:06.204 INFO:teuthology.orchestra.run.smithi159.stdout:3836 trunc from 0xe4bf8b3 to 0x73a3ef6 2024-09-17T22:58:06.206 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:06.204+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:06.221 INFO:teuthology.orchestra.run.smithi159.stdout:3837 trunc from 0x73a3ef6 to 0x565e34e 2024-09-17T22:58:06.238 INFO:teuthology.orchestra.run.smithi159.stdout:3838 punch from 0x2081573 to 0x20884a5, (0x6f32 bytes) 2024-09-17T22:58:06.249 INFO:teuthology.orchestra.run.smithi159.stdout:3840 writesame 0x9bc8635 thru 0x9bd68b4 data_size 0x97(0xe280 bytes) 2024-09-17T22:58:06.249 INFO:teuthology.orchestra.run.smithi159.stdout:3841 write 0x5d5d512 thru 0x5d658cd (0x83bc bytes) 2024-09-17T22:58:06.251 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:06.251 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:06.251 INFO:teuthology.orchestra.run.smithi159.stdout:3842 read 0x4c63995 thru 0x4c7079d (0xce09 bytes) 2024-09-17T22:58:06.252 INFO:teuthology.orchestra.run.smithi159.stdout:3843 read 0x7e6779 thru 0x7f3460 (0xcce8 bytes) 2024-09-17T22:58:06.252 INFO:teuthology.orchestra.run.smithi159.stdout:3845 writesame 0xc324c5 thru 0xc418b0 data_size 0x86(0xf3ec bytes) 2024-09-17T22:58:06.254 INFO:teuthology.orchestra.run.smithi159.stdout:3847 write 0x27dfddc thru 0x27e5100 (0x5325 bytes) 2024-09-17T22:58:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:3849 write 0xb9d4c4 thru 0xba6050 (0x8b8d bytes) 2024-09-17T22:58:06.260 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:06.269 INFO:teuthology.orchestra.run.smithi159.stdout:3851 write 0xdf58dfd thru 0xdf5de21 (0x5025 bytes) 2024-09-17T22:58:06.272 INFO:teuthology.orchestra.run.smithi159.stdout:3852 punch from 0x2df8306 to 0x2e07ceb, (0xf9e5 bytes) 2024-09-17T22:58:06.272 INFO:teuthology.orchestra.run.smithi159.stdout:3853 write 0x8c06ea thru 0x8c087c (0x193 bytes) 2024-09-17T22:58:06.274 INFO:teuthology.orchestra.run.smithi159.stdout:3854 write 0xc7517f9 thru 0xc75d91d (0xc125 bytes) 2024-09-17T22:58:06.276 INFO:teuthology.orchestra.run.smithi159.stdout:3855 write 0xd020115 thru 0xd021f6d (0x1e59 bytes) 2024-09-17T22:58:06.278 INFO:teuthology.orchestra.run.smithi159.stdout:3857 read 0x8359bf2 thru 0x83602d3 (0x66e2 bytes) 2024-09-17T22:58:06.279 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:06.279 INFO:teuthology.orchestra.run.smithi159.stdout:3859 punch from 0x913d6e1 to 0x9143eab, (0x67ca bytes) 2024-09-17T22:58:06.279 INFO:teuthology.orchestra.run.smithi159.stdout:3860 trunc from 0xdf5de22 to 0xe4761c6 2024-09-17T22:58:06.290 INFO:teuthology.orchestra.run.smithi159.stdout:3861 trunc from 0xe4761c6 to 0x9a3a0d2 2024-09-17T22:58:06.301 INFO:teuthology.orchestra.run.smithi159.stdout:3862 write 0x22121a3 thru 0x221e5c1 (0xc41f bytes) 2024-09-17T22:58:06.304 INFO:teuthology.orchestra.run.smithi159.stdout:3865 write 0x8dd19e4 thru 0x8dd933d (0x795a bytes) 2024-09-17T22:58:06.307 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:06.305+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:06.314 INFO:teuthology.orchestra.run.smithi159.stdout:3866 writesame 0xa45eddb thru 0xa46e23d data_size 0x1(0xf463 bytes) 2024-09-17T22:58:06.314 INFO:teuthology.orchestra.run.smithi159.stdout:3869 write 0x30aeaec thru 0x30af7b1 (0xcc6 bytes) 2024-09-17T22:58:06.325 INFO:teuthology.orchestra.run.smithi159.stdout:3870 read 0x3a54857 thru 0x3a625d4 (0xdd7e bytes) 2024-09-17T22:58:06.334 INFO:teuthology.orchestra.run.smithi159.stdout:3872 write 0xd021c17 thru 0xd025af0 (0x3eda bytes) 2024-09-17T22:58:06.339 INFO:teuthology.orchestra.run.smithi159.stdout:3873 punch from 0x6bac1e7 to 0x6bb3588, (0x73a1 bytes) 2024-09-17T22:58:06.339 INFO:teuthology.orchestra.run.smithi159.stdout:3876 read 0x7ddde44 thru 0x7de16f9 (0x38b6 bytes) 2024-09-17T22:58:06.339 INFO:teuthology.orchestra.run.smithi159.stdout:3877 write 0x22be173 thru 0x22c7967 (0x97f5 bytes) 2024-09-17T22:58:06.340 INFO:teuthology.orchestra.run.smithi159.stdout:3879 write 0x7d656d0 thru 0x7d675d3 (0x1f04 bytes) 2024-09-17T22:58:06.344 INFO:teuthology.orchestra.run.smithi159.stdout:3880 read 0x604f157 thru 0x604ff26 (0xdd0 bytes) 2024-09-17T22:58:06.344 INFO:teuthology.orchestra.run.smithi159.stdout:3881 punch from 0xae654db to 0xae6dbfd, (0x8722 bytes) 2024-09-17T22:58:06.344 INFO:teuthology.orchestra.run.smithi159.stdout:3882 read 0x8c6764d thru 0x8c74fcf (0xd983 bytes) 2024-09-17T22:58:06.344 INFO:teuthology.orchestra.run.smithi159.stdout:3885 read 0xcabbe59 thru 0xcac6376 (0xa51e bytes) 2024-09-17T22:58:06.344 INFO:teuthology.orchestra.run.smithi159.stdout:3886 punch from 0x4067ebd to 0x406f993, (0x7ad6 bytes) 2024-09-17T22:58:06.345 INFO:teuthology.orchestra.run.smithi159.stdout:3888 writesame 0x94a5d54 thru 0x94b06e7 data_size 0x2(0xa994 bytes) 2024-09-17T22:58:06.368 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:06.368 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:06.369 INFO:teuthology.orchestra.run.smithi159.stdout:3889 trunc from 0xd025af1 to 0x17debc2 2024-09-17T22:58:06.378 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:06.416 INFO:teuthology.orchestra.run.smithi159.stdout:3891 writesame 0xc49fa84 thru 0xc4a9426 data_size 0x1(0x99a3 bytes) 2024-09-17T22:58:06.416 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:06.414+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:06.423 INFO:teuthology.orchestra.run.smithi159.stdout:3892 trunc from 0xc4a9427 to 0x638ac75 2024-09-17T22:58:06.428 INFO:teuthology.orchestra.run.smithi159.stdout:3893 punch from 0xf24fd to 0xf4498, (0x1f9b bytes) 2024-09-17T22:58:06.434 INFO:teuthology.orchestra.run.smithi159.stdout:3895 write 0x7496d44 thru 0x74a16c9 (0xa986 bytes) 2024-09-17T22:58:06.438 INFO:teuthology.orchestra.run.smithi159.stdout:3896 read 0x19159f5 thru 0x1916734 (0xd40 bytes) 2024-09-17T22:58:06.441 INFO:teuthology.orchestra.run.smithi159.stdout:3898 trunc from 0x74a16ca to 0x4de0f9f 2024-09-17T22:58:06.451 INFO:teuthology.orchestra.run.smithi159.stdout:3901 read 0x27070e thru 0x2744ac (0x3d9f bytes) 2024-09-17T22:58:06.466 INFO:teuthology.orchestra.run.smithi159.stdout:3904 write 0xcfed942 thru 0xcff21b8 (0x4877 bytes) 2024-09-17T22:58:06.470 INFO:teuthology.orchestra.run.smithi159.stdout:3905 trunc from 0xcff21b9 to 0xc491b25 2024-09-17T22:58:06.479 INFO:teuthology.orchestra.run.smithi159.stdout:3906 writesame 0xbac78e4 thru 0xbacd34e data_size 0x1(0x5a6b bytes) 2024-09-17T22:58:06.492 INFO:teuthology.orchestra.run.smithi159.stdout:3909 read 0x539b5b8 thru 0x53a8480 (0xcec9 bytes) 2024-09-17T22:58:06.492 INFO:teuthology.orchestra.run.smithi159.stdout:3912 write 0x9c8ea67 thru 0x9c9961b (0xabb5 bytes) 2024-09-17T22:58:06.496 INFO:teuthology.orchestra.run.smithi159.stdout:3913 write 0x2af79f5 thru 0x2b05dec (0xe3f8 bytes) 2024-09-17T22:58:06.498 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:06.498 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:06.499 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:06.508 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:06.507+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:06.510 INFO:teuthology.orchestra.run.smithi159.stdout:3916 trunc from 0xc491b25 to 0x27d6f6f 2024-09-17T22:58:06.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:06.524 INFO:teuthology.orchestra.run.smithi159.stdout:3918 read 0x17fe39f thru 0x180a54a (0xc1ac bytes) 2024-09-17T22:58:06.536 INFO:teuthology.orchestra.run.smithi159.stdout:3919 write 0x815d021 thru 0x8162a60 (0x5a40 bytes) 2024-09-17T22:58:06.539 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:06.546 INFO:teuthology.orchestra.run.smithi159.stdout:3921 trunc from 0x8162a61 to 0xe2970b0 2024-09-17T22:58:06.551 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:06.551 INFO:teuthology.orchestra.run.smithi159.stdout:3923 read 0xcacfe7c thru 0xcad0308 (0x48d bytes) 2024-09-17T22:58:06.551 INFO:teuthology.orchestra.run.smithi159.stdout:3924 write 0x830188 thru 0x832315 (0x218e bytes) 2024-09-17T22:58:06.552 INFO:teuthology.orchestra.run.smithi159.stdout:3925 read 0x4f4236e thru 0x4f4e07c (0xbd0f bytes) 2024-09-17T22:58:06.553 INFO:teuthology.orchestra.run.smithi159.stdout:3926 write 0x9abc1d2 thru 0x9abfb24 (0x3953 bytes) 2024-09-17T22:58:06.555 INFO:teuthology.orchestra.run.smithi159.stdout:3927 write 0x29ba6d3 thru 0x29bfb12 (0x5440 bytes) 2024-09-17T22:58:06.558 INFO:teuthology.orchestra.run.smithi159.stdout:3931 write 0x475ab54 thru 0x475bb28 (0xfd5 bytes) 2024-09-17T22:58:06.559 INFO:teuthology.orchestra.run.smithi159.stdout:3932 writesame 0xdd8a263 thru 0xdd8a585 data_size 0x1(0x323 bytes) 2024-09-17T22:58:06.562 INFO:teuthology.orchestra.run.smithi159.stdout:3933 read 0x751762b thru 0x752691a (0xf2f0 bytes) 2024-09-17T22:58:06.571 INFO:teuthology.orchestra.run.smithi159.stdout:3934 trunc from 0xe2970b0 to 0x730751a 2024-09-17T22:58:06.585 INFO:teuthology.orchestra.run.smithi159.stdout:3935 read 0xbc5a88 thru 0xbc66b2 (0xc2b bytes) 2024-09-17T22:58:06.585 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:06.585 INFO:teuthology.orchestra.run.smithi159.stdout:3937 punch from 0x50aaa0e to 0x50b5f99, (0xb58b bytes) 2024-09-17T22:58:06.586 INFO:teuthology.orchestra.run.smithi159.stdout:3938 clone 27 order 21 su 32768 sc 7 2024-09-17T22:58:06.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:06.609 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:06.607+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:06.638 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:06.638 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:06.647 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:06.709 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:06.707+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:06.714 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:06.715 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:06.732 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:06.732 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:06.748 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:06.748 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486952932000001, 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:58:06.755 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:06.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486848896000001, 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:58:06.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:58:06.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486827635000001, 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.768 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:06.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486806292000001, 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.772 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:58:06.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.776 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486736834999999, 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:58:06.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486717856999999, 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.781 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:06.784 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486688928, 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.785 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:58:06.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486662989999999, 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:58:06.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.794 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486634995999999, 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:58:06.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:58:06.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486614071999998, 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.799 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:06.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:58:06.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486567331, 2024-09-17T22:58:06.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:06.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:58:06.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486544923, 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:58:06.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:06.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486521579000001, 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.812 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:58:06.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.48649889, 2024-09-17T22:58:06.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:06.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:58:06.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486476691, 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:58:06.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:58:06.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486447904999999, 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.825 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:06.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:58:06.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.829 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486425272000002, 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:58:06.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486406019, 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:58:06.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:58:06.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:58:06.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 30.486378843000001, 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.838 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:06.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 22.308739847999998, 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:06.842 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:06.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:06.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:06.849 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:06.849 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:06.850 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:06.851 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:06.851 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:06.852 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:06.852 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:06.852 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:06.852 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:06.853 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:06.853 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:06.854 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:06.847+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:06.887 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:58:06.949 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone26 from 0x730751a (overlap 0x17debc2) to 0x4c625a 2024-09-17T22:58:06.950 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:06.948+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:07.007 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:07.007 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.016 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:07.019 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T22:58:07.060 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:07.058+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:07.141 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:07.141 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:07.142 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:07.142 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:07.192 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:07.189+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:07.253 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:07.251+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:07.295 INFO:teuthology.orchestra.run.smithi159.stdout:3940 read 0x2530bb3 thru 0x2533c34 (0x3082 bytes) 2024-09-17T22:58:07.296 INFO:teuthology.orchestra.run.smithi159.stdout:3941 punch from 0x1b05a13 to 0x1b0fee4, (0xa4d1 bytes) 2024-09-17T22:58:07.300 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.300 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:07.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:07.300 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:07.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16134 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.042065342000001, 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036277120000000003, 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16134 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255128+0000", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255129+0000", 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:58:07.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255131+0000", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.916e-06 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255137+0000", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3509999999999998e-06 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255142+0000", 2024-09-17T22:58:07.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.882e-06 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291253+0000", 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036111153 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291337+0000", 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3535000000000006e-05 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291405+0000", 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8858999999999999e-05 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16135 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.042045369, 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036429299999999998, 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16135 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255148+0000", 2024-09-17T22:58:07.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255149+0000", 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255151+0000", 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255157+0000", 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255162+0000", 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291437+0000", 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036274726 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291514+0000", 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7342000000000006e-05 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291578+0000", 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3634999999999996e-05 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16159 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.042025630000001, 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036685328000000003, 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16159 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.310 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255168+0000", 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:58:07.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255170+0000", 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255177+0000", 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4989999999999999e-06 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255193+0000", 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5909999999999998e-05 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291665+0000", 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036471816999999997 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291751+0000", 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6509999999999997e-05 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291853+0000", 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102099 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16161 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:07.314 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041988078999999, 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036865504, 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16161 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255206+0000", 2024-09-17T22:58:07.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255208+0000", 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3250000000000002e-06 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255218+0000", 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4080000000000004e-06 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255223+0000", 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291884+0000", 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036661457000000001 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.291966+0000", 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1920000000000002e-05 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292071+0000", 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000104608 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16163 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041963629000001, 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037090170999999998, 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.319 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16163 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:07.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255230+0000", 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255233+0000", 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255242+0000", 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0750000000000004e-06 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255247+0000", 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:07.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292101+0000", 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036854162000000003 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292174+0000", 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2805000000000005e-05 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292320+0000", 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014646799999999999 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16165 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1728512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041940190999998, 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037305086000000001, 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.326 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16165 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255253+0000", 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255254+0000", 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255256+0000", 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.12e-06 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255265+0000", 2024-09-17T22:58:07.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8959999999999992e-06 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255270+0000", 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8760000000000004e-06 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292365+0000", 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037095377999999998 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292452+0000", 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6947000000000006e-05 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292558+0000", 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010643 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16167 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.331 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041917155, 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037507133999999998, 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16167 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255276+0000", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255277+0000", 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.46e-07 2024-09-17T22:58:07.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255279+0000", 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255288+0000", 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8429999999999999e-06 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255293+0000", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292603+0000", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037309869000000002 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292681+0000", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8484000000000003e-05 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292784+0000", 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010224 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16169 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041894134, 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037775437000000002, 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.336 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16169 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255299+0000", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255300+0000", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255302+0000", 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255308+0000", 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255313+0000", 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292863+0000", 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037549224999999999 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.292969+0000", 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010604500000000001 2024-09-17T22:58:07.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293075+0000", 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010620699999999999 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16171 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1875968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:07.341 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041874239999999, 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037958219000000001, 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16171 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255319+0000", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255320+0000", 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2899999999999999e-07 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255322+0000", 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.906e-06 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255337+0000", 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5452999999999999e-05 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255343+0000", 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293106+0000", 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037763707000000001 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293186+0000", 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0153000000000003e-05 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293278+0000", 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1114999999999998e-05 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16172 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1925120~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.346 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041843957000001, 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038145196999999999, 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16172 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255350+0000", 2024-09-17T22:58:07.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8400000000000004e-07 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255359+0000", 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255367+0000", 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0649999999999999e-06 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255372+0000", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1390000000000003e-06 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293318+0000", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037945566 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293399+0000", 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1168999999999995e-05 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293495+0000", 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6162000000000005e-05 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16173 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041812761999999, 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038380816999999998, 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.351 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16173 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255381+0000", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1699999999999998e-07 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255388+0000", 2024-09-17T22:58:07.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2079999999999997e-06 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255402+0000", 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4532000000000001e-05 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255408+0000", 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7740000000000001e-06 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293577+0000", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038169526000000002 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293664+0000", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6312999999999998e-05 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293762+0000", 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7947000000000003e-05 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.355 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16174 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041778006000001, 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038531986999999997, 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.356 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16174 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255416+0000", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255418+0000", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255426+0000", 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1620000000000002e-06 2024-09-17T22:58:07.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255436+0000", 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0851e-05 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293804+0000", 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038367215000000003 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293886+0000", 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2189999999999997e-05 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293948+0000", 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1709000000000002e-05 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.360 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16176 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.041750053000001, 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038738529000000001, 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16176 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.361 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255444+0000", 2024-09-17T22:58:07.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255446+0000", 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255453+0000", 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8170000000000001e-06 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.255458+0000", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.293991+0000", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038532376 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294084+0000", 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3614000000000002e-05 2024-09-17T22:58:07.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294182+0000", 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7949999999999996e-05 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16239 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.365 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.038377594, 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036338338999999997, 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16239 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258816+0000", 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:58:07.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258819+0000", 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3e-06 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258825+0000", 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258830+0000", 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.294968+0000", 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036137423000000002 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295061+0000", 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3177999999999995e-05 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295154+0000", 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3645000000000002e-05 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16241 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1835008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.038353028, 2024-09-17T22:58:07.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036566098999999998, 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16241 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:07.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258841+0000", 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4770000000000002e-06 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258850+0000", 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5490000000000003e-06 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258855+0000", 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.904e-06 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295247+0000", 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036392378000000003 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295334+0000", 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7058999999999994e-05 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295407+0000", 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2287999999999996e-05 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.375 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16243 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.038331162999999, 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036705072999999998, 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16243 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:07.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258862+0000", 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258863+0000", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.39e-07 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258866+0000", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.678e-06 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258872+0000", 2024-09-17T22:58:07.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.258877+0000", 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9810000000000003e-06 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295445+0000", 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036567582000000001 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295506+0000", 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1895000000000003e-05 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295568+0000", 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1085000000000002e-05 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.383 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16245 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1933312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.038004576999999, 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036671423000000002, 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16245 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259189+0000", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259190+0000", 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6300000000000004e-07 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259194+0000", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259205+0000", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0468e-05 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259212+0000", 2024-09-17T22:58:07.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1380000000000001e-06 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295666+0000", 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036453700999999998 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295753+0000", 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7321e-05 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295860+0000", 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010761599999999999 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16246 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.037970019999999, 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036882379, 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.388 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16246 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259224+0000", 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6599999999999996e-07 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259227+0000", 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259234+0000", 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8539999999999999e-06 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259239+0000", 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:58:07.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295903+0000", 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036664018999999999 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.295993+0000", 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9944000000000005e-05 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296106+0000", 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113158 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16248 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.037947849999998, 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037070651000000003, 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.393 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16248 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:07.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259246+0000", 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259248+0000", 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259254+0000", 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259260+0000", 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4489999999999998e-06 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296156+0000", 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036896089999999999 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296238+0000", 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2616999999999999e-05 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296316+0000", 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7964000000000001e-05 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16249 3.19 3:9a3def6a:::rbd_data.1188b0ea165c.0000000000000000:a [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.037776961999999, 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037100148999999999, 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16249 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259417+0000", 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8700000000000005e-07 2024-09-17T22:58:07.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259421+0000", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0470000000000004e-06 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259431+0000", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4190000000000008e-06 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.259437+0000", 2024-09-17T22:58:07.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4520000000000002e-06 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296352+0000", 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036914354000000003 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296427+0000", 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5603000000000001e-05 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.296517+0000", 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9586999999999995e-05 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.402 INFO:teuthology.orchestra.run.smithi159.stdout:3942 writesame 0x6b98303 thru 0x6ba75c2 data_size 0x20(0xf2c0 bytes) 2024-09-17T22:58:07.403 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:07.404 INFO:teuthology.orchestra.run.smithi159.stdout:3943 flatten 2024-09-17T22:58:07.438 INFO:teuthology.orchestra.run.smithi159.stdout:3944 trunc from 0x730751a to 0xbe45e7c 2024-09-17T22:58:07.438 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:07.435+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:07.443 INFO:teuthology.orchestra.run.smithi159.stdout:3945 punch from 0xbdfeeb7 to 0xbe0c03b, (0xd184 bytes) 2024-09-17T22:58:07.443 INFO:teuthology.orchestra.run.smithi159.stdout:3946 read 0x9003273 thru 0x9007fa4 (0x4d32 bytes) 2024-09-17T22:58:07.444 INFO:teuthology.orchestra.run.smithi159.stdout:3947 punch from 0xab95955 to 0xab9abd7, (0x5282 bytes) 2024-09-17T22:58:07.444 INFO:teuthology.orchestra.run.smithi159.stdout:3948 read 0x1b5602 thru 0x1bc02b (0x6a2a bytes) 2024-09-17T22:58:07.444 INFO:teuthology.orchestra.run.smithi159.stdout:3950 write 0x8e68c84 thru 0x8e73633 (0xa9b0 bytes) 2024-09-17T22:58:07.454 INFO:teuthology.orchestra.run.smithi159.stdout:3951 writesame 0xcf68436 thru 0xcf6e224 data_size 0x1(0x5def bytes) 2024-09-17T22:58:07.455 INFO:teuthology.orchestra.run.smithi159.stdout:3953 punch from 0x2c57f7f to 0x2c66892, (0xe913 bytes) 2024-09-17T22:58:07.455 INFO:teuthology.orchestra.run.smithi159.stdout:3954 write 0x1fec49c thru 0x1ffa66a (0xe1cf bytes) 2024-09-17T22:58:07.458 INFO:teuthology.orchestra.run.smithi159.stdout:3955 clone 28 order 19 su 262144 sc 12 2024-09-17T22:58:07.506 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:07.503+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:07.543 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:07.543 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.552 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:07.606 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:07.604+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:07.698 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:07.698 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.707 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:07.708 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:07.705+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:07.809 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:07.807+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:07.810 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4275.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 57.916562048000003, 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0049662040000000001, 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:58:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890980+0000", 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6899999999999999e-07 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890986+0000", 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.890997+0000", 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1403000000000001e-05 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891209+0000", 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000212208 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891283+0000", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3143000000000001e-05 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.891315+0000", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1878000000000002e-05 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895904+0000", 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045897680000000001 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:09.895946+0000", 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1770000000000002e-05 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1044 3.9 e28/22 3:954c0790:::rbd_data.118840f89003.0000000000000090:head v 28'4, mlcod=25'2)", 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.778760343999998, 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032794199999999999, 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.815 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028782+0000", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.09e-07 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028800+0000", 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7649999999999999e-05 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028812+0000", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1865000000000001e-05 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028819+0000", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7750000000000004e-06 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028870+0000", 2024-09-17T22:58:07.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0949000000000003e-05 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028889+0000", 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9199999999999999e-05 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032019+0000", 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031302230000000001 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032061+0000", 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1949000000000001e-05 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 41 entries 28'5 (0'0) error 3:9656af7d:::rbd_data.118840f89003.0000000000000092:head by client.4488.0:1048 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.778687196999996, 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034247520000000001, 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.820 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.821 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:07.821 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028855+0000", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028863+0000", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7249999999999997e-06 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028879+0000", 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5862000000000002e-05 2024-09-17T22:58:07.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.028892+0000", 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3076e-05 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.029115+0000", 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000223427 2024-09-17T22:58:07.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.032279+0000", 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.003164189 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.9 epoch 28/22 rep_tid 50 entries 28'7 (0'0) error 3:975adb25:::rbd_data.118840f89003.00000000000000a5:head by client.4488.0:1110 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'5)", 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.768085071000002, 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0077516210000000002, 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039457+0000", 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039458+0000", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.99e-07 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039466+0000", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4419999999999998e-06 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039486+0000", 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0523999999999999e-05 2024-09-17T22:58:07.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039502+0000", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5930000000000002e-05 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.039590+0000", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8012999999999999e-05 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047208+0000", 2024-09-17T22:58:07.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0076180129999999999 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:1121 3.4 e28/22 3:218d64de:::rbd_data.118840f89003.00000000000000a2:head v 28'6, mlcod=27'4)", 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.764982365999998, 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.004866296, 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.831 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042560+0000", 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042566+0000", 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042575+0000", 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3349999999999996e-06 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042580+0000", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042623+0000", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3396000000000003e-05 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.042644+0000", 2024-09-17T22:58:07.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1155999999999999e-05 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047379+0000", 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0047352699999999998 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:26.047426+0000", 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6424999999999997e-05 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:6774 3.0 e33/22 3:05c44946:::rbd_data.118846a2cbc0.0000000000000045:head v 33'36, mlcod=32'33)", 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 36.732097318999998, 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0053547780000000001, 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:58:07.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075445+0000", 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0299999999999998e-07 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075451+0000", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075468+0000", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7212999999999999e-05 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075474+0000", 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3899999999999998e-06 2024-09-17T22:58:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075522+0000", 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8114e-05 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.075543+0000", 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0389000000000001e-05 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080753+0000", 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0052103440000000004 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.080799+0000", 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6335999999999998e-05 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.840 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:7337 3.11 e33/28 3:89c4a1e2:::rbd_header.118846a2cbc0:head v 33'39, mlcod=33'36)", 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 36.612253705999997, 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0046186769999999999, 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195288+0000", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195289+0000", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195293+0000", 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.155e-06 2024-09-17T22:58:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195301+0000", 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8520000000000004e-06 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195307+0000", 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195340+0000", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3090999999999999e-05 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.195353+0000", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2757e-05 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199876+0000", 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0045229500000000004 2024-09-17T22:58:07.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:31.199907+0000", 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1340000000000001e-05 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22674 3.0 e45/22 3:000cde75:::rbd_data.1188903913c1.000000000000001f:head v 45'77, mlcod=44'74)", 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.667598289000001, 2024-09-17T22:58:07.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0062289370000000004, 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139944+0000", 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139953+0000", 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0229999999999998e-06 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139962+0000", 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0699999999999996e-06 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.139968+0000", 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140030+0000", 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1803000000000004e-05 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.140046+0000", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6093999999999999e-05 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146129+0000", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0060834239999999996 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.146173+0000", 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3238999999999997e-05 2024-09-17T22:58:07.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "pg_update_log_missing(3.0 epoch 45/22 rep_tid 301 entries 45'78 (0'0) error 3:047d1ef5:::rbd_data.1188903913c1.0000000000000029:head by client.4488.0:22684 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 44'74)", 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.665900051000001, 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.007471218, 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141642+0000", 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.44e-07 2024-09-17T22:58:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141651+0000", 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4449999999999996e-06 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141671+0000", 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0373e-05 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141695+0000", 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4224e-05 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.141782+0000", 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6514999999999994e-05 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149113+0000", 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0073311169999999998 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22687 3.4 e45/22 3:25296614:::rbd_data.1188903913c1.000000000000002b:head v 45'156, mlcod=44'152)", 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.665026437000002, 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0066082959999999996, 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:58:07.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142515+0000", 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142516+0000", 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3900000000000004e-07 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142523+0000", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7429999999999996e-06 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142536+0000", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3151e-05 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142543+0000", 2024-09-17T22:58:07.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7100000000000001e-06 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142607+0000", 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3765999999999999e-05 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.142627+0000", 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.035e-05 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149088+0000", 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0064607140000000002 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.149124+0000", 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6123e-05 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:22907 3.4 e45/22 3:244fb18a:::rbd_data.1188903913c1.00000000000001f6:head v 45'158, mlcod=45'154)", 2024-09-17T22:58:07.858 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.648453524000001, 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031838320000000002, 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159088+0000", 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159089+0000", 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5499999999999998e-07 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159095+0000", 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4409999999999998e-06 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159108+0000", 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.258e-05 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159115+0000", 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9419999999999998e-06 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159210+0000", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4853999999999997e-05 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.159231+0000", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1583e-05 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162234+0000", 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030021539999999999 2024-09-17T22:58:07.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.162272+0000", 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8622999999999999e-05 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:39761 3.4 e54/22 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:head v 54'167, mlcod=54'165)", 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.988510049, 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031656729999999999, 2024-09-17T22:58:07.863 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819032+0000", 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819033+0000", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.562e-06 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819756+0000", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00072241600000000003 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819777+0000", 2024-09-17T22:58:07.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1429000000000001e-05 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819787+0000", 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0137e-05 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819890+0000", 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102441 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.819913+0000", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3529000000000001e-05 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822139+0000", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022256530000000002 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:53.822198+0000", 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8505999999999999e-05 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.867 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:42197 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.762098395000001, 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032112489999999998, 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 42197 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045443+0000", 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045445+0000", 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.615e-06 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045452+0000", 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8589999999999999e-06 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045463+0000", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1001e-05 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045481+0000", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8423e-05 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045537+0000", 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6069999999999997e-05 2024-09-17T22:58:07.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.045626+0000", 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8351999999999997e-05 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048535+0000", 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029092060000000001 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048589+0000", 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3761999999999998e-05 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.048655+0000", 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5961000000000001e-05 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44169 3.11 3:8b5f814e:::rbd_data.118894cf7479.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:58:07.873 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.571654296, 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0042738230000000004, 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44169 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235888+0000", 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235890+0000", 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0789999999999999e-06 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235900+0000", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0277e-05 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235923+0000", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2693000000000001e-05 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.235933+0000", 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0441e-05 2024-09-17T22:58:07.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236005+0000", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1957000000000002e-05 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.236091+0000", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5718999999999997e-05 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240059+0000", 2024-09-17T22:58:07.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0039685099999999997 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240096+0000", 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6431999999999999e-05 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.240161+0000", 2024-09-17T22:58:07.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5715000000000004e-05 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44203 3.0 e54/22 3:037ebc53:::rbd_data.118894cf7479.000000000000003a:head v 54'130, mlcod=54'128)", 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.535578162, 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0035646229999999998, 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.879 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271964+0000", 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.880 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271965+0000", 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1739999999999999e-06 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271973+0000", 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8650000000000001e-06 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271986+0000", 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2899e-05 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.271993+0000", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2520000000000002e-06 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272061+0000", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392000000000005e-05 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272083+0000", 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2161e-05 2024-09-17T22:58:07.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275498+0000", 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034148059999999998 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275528+0000", 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0074e-05 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:44204 3.4 e54/22 3:23e33dd1:::rbd_data.118894cf7479.000000000000003b:head v 54'172, mlcod=54'170)", 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.53551296, 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0034598849999999998, 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.884 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272029+0000", 2024-09-17T22:58:07.885 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.01e-07 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272034+0000", 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6179999999999997e-06 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272041+0000", 2024-09-17T22:58:07.886 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4290000000000004e-06 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272047+0000", 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3890000000000004e-06 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272096+0000", 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9033000000000003e-05 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.272114+0000", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8158000000000001e-05 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275444+0000", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033300489999999999 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.275489+0000", 2024-09-17T22:58:07.888 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4808000000000002e-05 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:44658 3.11 3:8a68e66c:::rbd_data.118894cf7479.000000000000007f:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.369219779, 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0033015309999999999, 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.889 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 44658 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438322+0000", 2024-09-17T22:58:07.890 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438324+0000", 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.153e-06 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438332+0000", 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7759999999999997e-06 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.891 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438349+0000", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7005999999999999e-05 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438359+0000", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0186999999999999e-05 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438420+0000", 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0915000000000002e-05 2024-09-17T22:58:07.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.438517+0000", 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6848999999999994e-05 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441513+0000", 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002996383 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.893 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441545+0000", 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1656999999999999e-05 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.441624+0000", 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8604999999999997e-05 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:45020 3.11 3:8ceaf2d8:::rbd_data.118894cf7479.00000000000000d2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:58:07.894 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 13.113359159, 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0036221339999999999, 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 45020 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:58:07.895 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694183+0000", 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694185+0000", 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.896 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694195+0000", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0603e-05 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694222+0000", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6573000000000001e-05 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694236+0000", 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3816e-05 2024-09-17T22:58:07.897 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694368+0000", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013227200000000001 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.694508+0000", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014027700000000001 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697713+0000", 2024-09-17T22:58:07.898 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0032041769999999999 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697749+0000", 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6742999999999997e-05 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:54.697805+0000", 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5535999999999998e-05 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.899 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48071 3.11 3:8902b897:::rbd_data.1188a43cbaa8.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.9443840750000003, 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.017833759000000001, 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48071 2024-09-17T22:58:07.900 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863158+0000", 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.901 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863159+0000", 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2240000000000001e-06 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863166+0000", 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5989999999999998e-06 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863181+0000", 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5637e-05 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.902 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863189+0000", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0360000000000006e-06 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863256+0000", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6255999999999997e-05 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.863338+0000", 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2157000000000006e-05 2024-09-17T22:58:07.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880830+0000", 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.017492245 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880895+0000", 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4717999999999999e-05 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.904 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.880992+0000", 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6886999999999996e-05 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48127 3.11 3:8902b897:::rbd_data.1188a43cbaa8.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write 2850816~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.8726535119999994, 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0048067099999999996, 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:07.905 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48127 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:07.906 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934888+0000", 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934890+0000", 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934914+0000", 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3643999999999999e-05 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.907 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934927+0000", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3159000000000001e-05 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934933+0000", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8510000000000004e-06 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.934987+0000", 2024-09-17T22:58:07.908 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4497999999999999e-05 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.935080+0000", 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2577000000000005e-05 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939562+0000", 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0044823440000000001 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.909 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939620+0000", 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8019000000000001e-05 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:57.939695+0000", 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4688e-05 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:07.910 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:07.912 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:07.912 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:07.909+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:08.012 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:08.010+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:08.063 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:08.064 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:08.064 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:08.064 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:08.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:08.078 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T22:58:08.088 INFO:teuthology.orchestra.run.smithi159.stdout:3957 punch from 0x6435651 to 0x643c4f4, (0x6ea3 bytes) 2024-09-17T22:58:08.092 INFO:teuthology.orchestra.run.smithi159.stdout:3958 read 0x503dacf thru 0x5045950 (0x7e82 bytes) 2024-09-17T22:58:08.092 INFO:teuthology.orchestra.run.smithi159.stdout:3959 write 0x141a6b8 thru 0x1420979 (0x62c2 bytes) 2024-09-17T22:58:08.095 INFO:teuthology.orchestra.run.smithi159.stdout:3961 punch from 0x523ab64 to 0x5243415, (0x88b1 bytes) 2024-09-17T22:58:08.101 INFO:teuthology.orchestra.run.smithi159.stdout:3962 trunc from 0xcf6e225 to 0x7f6ec92 2024-09-17T22:58:08.113 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:08.110+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:08.139 INFO:teuthology.orchestra.run.smithi159.stdout:3963 write 0xe9696b5 thru 0xe96c22a (0x2b76 bytes) 2024-09-17T22:58:08.158 INFO:teuthology.orchestra.run.smithi159.stdout:3964 trunc from 0xe96c22b to 0x463f881 2024-09-17T22:58:08.189 INFO:teuthology.orchestra.run.smithi159.stdout:3966 read 0xb278f2 thru 0xb2a4c7 (0x2bd6 bytes) 2024-09-17T22:58:08.193 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.193 INFO:teuthology.orchestra.run.smithi159.stdout:3969 read 0x375971b thru 0x37687ba (0xf0a0 bytes) 2024-09-17T22:58:08.193 INFO:teuthology.orchestra.run.smithi159.stdout:3970 writesame 0x628f46 thru 0x62a660 data_size 0x23(0x171b bytes) 2024-09-17T22:58:08.200 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:08.200 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:08.200 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:08.201 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:08.201 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:08.201 INFO:teuthology.orchestra.run.smithi159.stdout:3971 trunc from 0x463f881 to 0x8aafcca 2024-09-17T22:58:08.212 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:08.218 INFO:teuthology.orchestra.run.smithi159.stdout:3974 write 0xdd3e230 thru 0xdd4dec5 (0xfc96 bytes) 2024-09-17T22:58:08.219 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:08.216+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:08.224 INFO:teuthology.orchestra.run.smithi159.stdout:3975 read 0xbf9e4d5 thru 0xbfab394 (0xcec0 bytes) 2024-09-17T22:58:08.240 INFO:teuthology.orchestra.run.smithi159.stdout:3976 trunc from 0xdd4dec6 to 0xf0cb8d 2024-09-17T22:58:08.291 INFO:teuthology.orchestra.run.smithi159.stdout:3977 write 0xe37a784 thru 0xe37e4dc (0x3d59 bytes) 2024-09-17T22:58:08.294 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.294 INFO:teuthology.orchestra.run.smithi159.stdout:3979 write 0xb15f01f thru 0xb162fa9 (0x3f8b bytes) 2024-09-17T22:58:08.297 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.297 INFO:teuthology.orchestra.run.smithi159.stdout:3982 writesame 0xcbadbc9 thru 0xcbb45e3 data_size 0x1(0x6a1b bytes) 2024-09-17T22:58:08.314 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:08.311+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:08.317 INFO:teuthology.orchestra.run.smithi159.stdout:3983 read 0x3066f17 thru 0x306cce0 (0x5dca bytes) 2024-09-17T22:58:08.317 INFO:teuthology.orchestra.run.smithi159.stdout:3984 writesame 0x281a2b9 thru 0x282349c data_size 0x2(0x91e4 bytes) 2024-09-17T22:58:08.328 INFO:teuthology.orchestra.run.smithi159.stdout:3985 write 0xbcc23c3 thru 0xbcc4f7b (0x2bb9 bytes) 2024-09-17T22:58:08.336 INFO:teuthology.orchestra.run.smithi159.stdout:3986 writesame 0xe54113a thru 0xe544fda data_size 0x1(0x3ea1 bytes) 2024-09-17T22:58:08.337 INFO:teuthology.orchestra.run.smithi159.stdout:3989 write 0x90eb72f thru 0x90f75f9 (0xbecb bytes) 2024-09-17T22:58:08.339 INFO:teuthology.orchestra.run.smithi159.stdout:3990 read 0x21c043b thru 0x21c445a (0x4020 bytes) 2024-09-17T22:58:08.340 INFO:teuthology.orchestra.run.smithi159.stdout:3991 write 0x47d6deb thru 0x47da25f (0x3475 bytes) 2024-09-17T22:58:08.342 INFO:teuthology.orchestra.run.smithi159.stdout:3992 punch from 0x3fc3142 to 0x3fccf93, (0x9e51 bytes) 2024-09-17T22:58:08.345 INFO:teuthology.orchestra.run.smithi159.stdout:3993 trunc from 0xe544fdb to 0xc10baaa 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5757 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760922368999999, 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038199444999999999, 2024-09-17T22:58:08.351 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5757 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5600000000000002e-07 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586464+0000", 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586468+0000", 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8630000000000001e-06 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586471+0000", 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7549999999999999e-06 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624560+0000", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038088960999999998 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624607+0000", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6740000000000003e-05 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624661+0000", 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4732999999999999e-05 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.355 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5758 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760909480000002, 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038315071999999999, 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.356 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5758 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:08.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5100000000000001e-07 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586476+0000", 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.113e-06 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586483+0000", 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5200000000000003e-06 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586487+0000", 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3479999999999997e-06 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624681+0000", 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038194371999999997 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624722+0000", 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0367999999999997e-05 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624790+0000", 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8100000000000002e-05 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5759 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3948544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760893754999998, 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038450864000000001, 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5759 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586491+0000", 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:58:08.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586492+0000", 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5349999999999999e-06 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586495+0000", 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586498+0000", 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6170000000000001e-06 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624828+0000", 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038329514000000002 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624875+0000", 2024-09-17T22:58:08.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7107999999999997e-05 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624941+0000", 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6647000000000001e-05 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5760 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760883047999997, 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038669935000000003, 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5760 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:58:08.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586503+0000", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0899999999999999e-06 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586506+0000", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586508+0000", 2024-09-17T22:58:08.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624980+0000", 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038472343999999999 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625076+0000", 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5214e-05 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625171+0000", 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5421999999999995e-05 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5761 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4276224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760873068000002, 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038805352000000001, 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.370 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5761 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.22e-07 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586512+0000", 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0210000000000001e-06 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586515+0000", 2024-09-17T22:58:08.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:58:08.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586518+0000", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625211+0000", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038693268000000003 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625261+0000", 2024-09-17T22:58:08.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9345000000000003e-05 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625317+0000", 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5850999999999998e-05 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5762 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4472832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:08.375 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760863176999997, 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038916981000000003, 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5762 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586525+0000", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.224e-06 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586532+0000", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3950000000000003e-06 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586535+0000", 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625339+0000", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038804584000000003 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625385+0000", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5099000000000003e-05 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625438+0000", 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3511999999999999e-05 2024-09-17T22:58:08.379 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5763 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4669440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760845246999999, 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039029253999999999, 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.380 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5763 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:08.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586541+0000", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.268e-06 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586547+0000", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586554+0000", 2024-09-17T22:58:08.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625465+0000", 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038910924 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625511+0000", 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5179999999999998e-05 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625568+0000", 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7738999999999997e-05 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.386 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5764 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4866048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760824395999997, 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039122912000000003, 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.387 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5764 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586561+0000", 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.266e-06 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586566+0000", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586568+0000", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625592+0000", 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039024174000000002 2024-09-17T22:58:08.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625634+0000", 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1464000000000001e-05 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625683+0000", 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9002000000000003e-05 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5765 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5062656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760812549000001, 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039240865999999999, 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5765 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586895+0000", 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000323658 2024-09-17T22:58:08.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586899+0000", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6260000000000001e-06 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586910+0000", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1239e-05 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586917+0000", 2024-09-17T22:58:08.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625710+0000", 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038792608999999999 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625757+0000", 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7416000000000001e-05 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625813+0000", 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5395000000000001e-05 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5766 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5259264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760456417, 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038992026999999999, 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5766 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:08.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586931+0000", 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586939+0000", 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2549999999999997e-06 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586945+0000", 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625828+0000", 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038882947000000001 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625871+0000", 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3204000000000002e-05 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625920+0000", 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8539999999999999e-05 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5767 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5455872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.402 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.760431326999999, 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039097260000000002, 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.403 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5767 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:08.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588020+0000", 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00106675 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588024+0000", 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588033+0000", 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588044+0000", 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1800000000000001e-05 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625943+0000", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037898695000000003 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625988+0000", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4577999999999999e-05 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626050+0000", 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2605000000000001e-05 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.407 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5769 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5849088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.759300695, 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038207729000000003, 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5769 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588086+0000", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.103e-06 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588094+0000", 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6459999999999993e-06 2024-09-17T22:58:08.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588099+0000", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1259999999999997e-06 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626190+0000", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.03809067 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626234+0000", 2024-09-17T22:58:08.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4051999999999998e-05 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626291+0000", 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7694e-05 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5770 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.759277722999997, 2024-09-17T22:58:08.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038303865999999999, 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5770 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588109+0000", 2024-09-17T22:58:08.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0899999999999999e-06 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588116+0000", 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2439999999999996e-06 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588122+0000", 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8640000000000001e-06 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626315+0000", 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038193105999999998 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626359+0000", 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3828000000000001e-05 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626410+0000", 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1328000000000001e-05 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.416 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5771 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.759255443000001, 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038402700999999997, 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.417 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5771 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588131+0000", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588138+0000", 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3049999999999998e-06 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588143+0000", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4650000000000002e-06 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626433+0000", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038289895999999997 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626476+0000", 2024-09-17T22:58:08.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2913000000000001e-05 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626532+0000", 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5692000000000001e-05 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5772 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6438912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.759233784999999, 2024-09-17T22:58:08.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038494234000000002, 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5772 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590017+0000", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018664980000000001 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590020+0000", 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4769999999999999e-06 2024-09-17T22:58:08.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590027+0000", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5209999999999999e-06 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590032+0000", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626554+0000", 2024-09-17T22:58:08.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036521551999999999 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626595+0000", 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1900000000000002e-05 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626645+0000", 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9205000000000001e-05 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5829 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.757029322999998, 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038295726000000002, 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5829 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.426 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592028+0000", 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001673379 2024-09-17T22:58:08.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592032+0000", 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5109999999999999e-06 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592038+0000", 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7379999999999999e-06 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592042+0000", 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628543+0000", 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036500828999999999 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628594+0000", 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1737999999999998e-05 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628651+0000", 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6320999999999998e-05 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5840 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.755178688000001, 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038232347, 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.430 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5840 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:08.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592208+0000", 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-06 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592214+0000", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7479999999999999e-06 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592220+0000", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.037e-06 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630310+0000", 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038090474999999999 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630373+0000", 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2467999999999999e-05 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630438+0000", 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5235999999999997e-05 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5841 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3375104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.434 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.755158450000003, 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038361579999999999, 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5841 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592228+0000", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592235+0000", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8040000000000004e-06 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592242+0000", 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2230000000000001e-06 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630468+0000", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038225982999999998 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630517+0000", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8656000000000003e-05 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630587+0000", 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0698000000000006e-05 2024-09-17T22:58:08.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5842 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.755135719999998, 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038469774999999998, 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5842 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594021+0000", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001772849 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594026+0000", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4399999999999998e-06 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594037+0000", 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1071e-05 2024-09-17T22:58:08.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594044+0000", 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9129999999999997e-06 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630613+0000", 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036569602999999999 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630659+0000", 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5246e-05 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630718+0000", 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9653e-05 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:08.443 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.193842579, 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:08.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:08.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:08.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:08.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:08.450 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:08.451 INFO:teuthology.orchestra.run.smithi159.stdout:3994 read 0xa60580a thru 0xa610327 (0xab1e bytes) 2024-09-17T22:58:08.451 INFO:teuthology.orchestra.run.smithi159.stdout:3999 punch from 0x3d866cc to 0x3d8b24b, (0x4b7f bytes) 2024-09-17T22:58:08.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:08.453 INFO:teuthology.orchestra.run.smithi159.stdout:4000 trunc from 0xc10baaa to 0x3888e77 2024-09-17T22:58:08.453 INFO:teuthology.orchestra.run.smithi159.stdout:4001 write 0x5de78b thru 0x5e61d6 (0x7a4c bytes) 2024-09-17T22:58:08.453 INFO:teuthology.orchestra.run.smithi159.stdout:4003 punch from 0x19d99b2 to 0x19db95d, (0x1fab bytes) 2024-09-17T22:58:08.453 INFO:teuthology.orchestra.run.smithi159.stdout:4004 punch from 0x18b6522 to 0x18b6739, (0x217 bytes) 2024-09-17T22:58:08.453 INFO:teuthology.orchestra.run.smithi159.stdout:4005 trunc from 0x3888e77 to 0x6e22fe8 2024-09-17T22:58:08.453 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:08.449+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:08.455 INFO:teuthology.orchestra.run.smithi159.stdout:4006 writesame 0x68f99d4 thru 0x68fe876 data_size 0x1(0x4ea3 bytes) 2024-09-17T22:58:08.467 INFO:teuthology.orchestra.run.smithi159.stdout:4008 write 0x57f89ec thru 0x5808487 (0xfa9c bytes) 2024-09-17T22:58:08.479 INFO:teuthology.orchestra.run.smithi159.stdout:4010 trunc from 0x6e22fe8 to 0x4b48556 2024-09-17T22:58:08.507 INFO:teuthology.orchestra.run.smithi159.stdout:4014 writesame 0x9eefa94 thru 0x9ef090f data_size 0x4(0xe7c bytes) 2024-09-17T22:58:08.508 INFO:teuthology.orchestra.run.smithi159.stdout:4016 read 0x5859956 thru 0x585b48f (0x1b3a bytes) 2024-09-17T22:58:08.508 INFO:teuthology.orchestra.run.smithi159.stdout:4017 punch from 0x5dc42ca to 0x5dd36cc, (0xf402 bytes) 2024-09-17T22:58:08.508 INFO:teuthology.orchestra.run.smithi159.stdout:4018 read 0x6bfe3d2 thru 0x6c0059c (0x21cb bytes) 2024-09-17T22:58:08.518 INFO:teuthology.orchestra.run.smithi159.stdout:4019 write 0xe7e0bbc thru 0xe7ea770 (0x9bb5 bytes) 2024-09-17T22:58:08.525 INFO:teuthology.orchestra.run.smithi159.stdout:4020 trunc from 0xe7ea771 to 0xb544ecb 2024-09-17T22:58:08.534 INFO:teuthology.orchestra.run.smithi159.stdout:4021 read 0x243bbd4 thru 0x2441721 (0x5b4e bytes) 2024-09-17T22:58:08.534 INFO:teuthology.orchestra.run.smithi159.stdout:4024 read 0x7ef64cc thru 0x7efd240 (0x6d75 bytes) 2024-09-17T22:58:08.534 INFO:teuthology.orchestra.run.smithi159.stdout:4025 read 0x7f3bf32 thru 0x7f40efd (0x4fcc bytes) 2024-09-17T22:58:08.534 INFO:teuthology.orchestra.run.smithi159.stdout:4026 read 0x4ef029a thru 0x4ef37b8 (0x351f bytes) 2024-09-17T22:58:08.534 INFO:teuthology.orchestra.run.smithi159.stdout:4027 write 0x1f52348 thru 0x1f5b620 (0x92d9 bytes) 2024-09-17T22:58:08.537 INFO:teuthology.orchestra.run.smithi159.stdout:4028 read 0xcbd88e thru 0xcbf9fc (0x216f bytes) 2024-09-17T22:58:08.538 INFO:teuthology.orchestra.run.smithi159.stdout:4030 punch from 0x9f2f6f to 0x9f52f6, (0x2387 bytes) 2024-09-17T22:58:08.538 INFO:teuthology.orchestra.run.smithi159.stdout:4031 read 0xa8b0cdd thru 0xa8b5016 (0x433a bytes) 2024-09-17T22:58:08.538 INFO:teuthology.orchestra.run.smithi159.stdout:4032 write 0x45b5d69 thru 0x45be372 (0x860a bytes) 2024-09-17T22:58:08.542 INFO:teuthology.orchestra.run.smithi159.stdout:4034 trunc from 0xb544ecb to 0xbb8fc74 2024-09-17T22:58:08.547 INFO:teuthology.orchestra.run.smithi159.stdout:4035 read 0x3b14b7c thru 0x3b17d35 (0x31ba bytes) 2024-09-17T22:58:08.547 INFO:teuthology.orchestra.run.smithi159.stdout:4036 writesame 0x43773d9 thru 0x43828ce data_size 0x15(0xb4f6 bytes) 2024-09-17T22:58:08.552 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:08.550+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:08.553 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.557 INFO:teuthology.orchestra.run.smithi159.stdout:4038 trunc from 0xbb8fc74 to 0x81761e8 2024-09-17T22:58:08.560 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:08.560 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:08.560 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:08.562 INFO:teuthology.orchestra.run.smithi159.stdout:4039 write 0x9ded5b thru 0x9e4382 (0x5628 bytes) 2024-09-17T22:58:08.568 INFO:teuthology.orchestra.run.smithi159.stdout:4041 punch from 0x6e5065e to 0x6e507ea, (0x18c bytes) 2024-09-17T22:58:08.569 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:08.576 INFO:teuthology.orchestra.run.smithi159.stdout:4042 writesame 0xaa65a2b thru 0xaa708d9 data_size 0x1(0xaeaf bytes) 2024-09-17T22:58:08.576 INFO:teuthology.orchestra.run.smithi159.stdout:4043 read 0x19a8742 thru 0x19abc89 (0x3548 bytes) 2024-09-17T22:58:08.581 INFO:teuthology.orchestra.run.smithi159.stdout:4044 writesame 0xb4d9b7d thru 0xb4e5dc5 data_size 0x3(0xc249 bytes) 2024-09-17T22:58:08.582 INFO:teuthology.orchestra.run.smithi159.stdout:4045 read 0x5b45293 thru 0x5b473fb (0x2169 bytes) 2024-09-17T22:58:08.582 INFO:teuthology.orchestra.run.smithi159.stdout:4046 write 0x87570ea thru 0x875d022 (0x5f39 bytes) 2024-09-17T22:58:08.585 INFO:teuthology.orchestra.run.smithi159.stdout:4047 read 0x75cdb92 thru 0x75d82a3 (0xa712 bytes) 2024-09-17T22:58:08.585 INFO:teuthology.orchestra.run.smithi159.stdout:4048 punch from 0x9d3b0f7 to 0x9d487c3, (0xd6cc bytes) 2024-09-17T22:58:08.585 INFO:teuthology.orchestra.run.smithi159.stdout:4049 read 0x1985d53 thru 0x198d707 (0x79b5 bytes) 2024-09-17T22:58:08.591 INFO:teuthology.orchestra.run.smithi159.stdout:4050 write 0xc769c29 thru 0xc779ba4 (0xff7c bytes) 2024-09-17T22:58:08.607 INFO:teuthology.orchestra.run.smithi159.stdout:4051 trunc from 0xc779ba5 to 0x139fc67 2024-09-17T22:58:08.617 INFO:teuthology.orchestra.run.smithi159.stdout:4052 read 0x9fa498 thru 0xa0056c (0x60d5 bytes) 2024-09-17T22:58:08.617 INFO:teuthology.orchestra.run.smithi159.stdout:4053 read 0xa348ac thru 0xa4008e (0xb7e3 bytes) 2024-09-17T22:58:08.618 INFO:teuthology.orchestra.run.smithi159.stdout:4056 punch from 0xe967a5 to 0xea02ab, (0x9b06 bytes) 2024-09-17T22:58:08.639 INFO:teuthology.orchestra.run.smithi159.stdout:4057 write 0xd2d0734 thru 0xd2d732d (0x6bfa bytes) 2024-09-17T22:58:08.641 INFO:teuthology.orchestra.run.smithi159.stdout:4058 write 0xa52b0ee thru 0xa5317a3 (0x66b6 bytes) 2024-09-17T22:58:08.645 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.645 INFO:teuthology.orchestra.run.smithi159.stdout:4060 write 0x2a79afb thru 0x2a84b1a (0xb020 bytes) 2024-09-17T22:58:08.653 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:08.651+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:08.662 INFO:teuthology.orchestra.run.smithi159.stdout:4062 trunc from 0xd2d732e to 0x2d66ba9 2024-09-17T22:58:08.676 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:08.677 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:08.677 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:08.677 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:08.677 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:08.685 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:08.688 INFO:teuthology.orchestra.run.smithi159.stdout:4063 write 0xa86085b thru 0xa8626cc (0x1e72 bytes) 2024-09-17T22:58:08.692 INFO:teuthology.orchestra.run.smithi159.stdout:4064 read 0xa152593 thru 0xa1603c3 (0xde31 bytes) 2024-09-17T22:58:08.692 INFO:teuthology.orchestra.run.smithi159.stdout:4065 write 0x37a1c61 thru 0x37a472d (0x2acd bytes) 2024-09-17T22:58:08.695 INFO:teuthology.orchestra.run.smithi159.stdout:4066 read 0x6c49a7c thru 0x6c51810 (0x7d95 bytes) 2024-09-17T22:58:08.695 INFO:teuthology.orchestra.run.smithi159.stdout:4070 punch from 0xaf992f to 0xb0143f, (0x7b10 bytes) 2024-09-17T22:58:08.695 INFO:teuthology.orchestra.run.smithi159.stdout:4071 read 0xa6217d2 thru 0xa62b72a (0x9f59 bytes) 2024-09-17T22:58:08.695 INFO:teuthology.orchestra.run.smithi159.stdout:4073 writesame 0x8868583 thru 0x8873bf7 data_size 0x1(0xb675 bytes) 2024-09-17T22:58:08.727 INFO:teuthology.orchestra.run.smithi159.stdout:4074 trunc from 0xa8626cd to 0x5288826 2024-09-17T22:58:08.739 INFO:teuthology.orchestra.run.smithi159.stdout:4075 read 0x20c8711 thru 0x20cfa37 (0x7327 bytes) 2024-09-17T22:58:08.739 INFO:teuthology.orchestra.run.smithi159.stdout:4076 writesame 0x358731 thru 0x36781a data_size 0x3(0xf0ea bytes) 2024-09-17T22:58:08.754 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:08.752+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:08.756 INFO:teuthology.orchestra.run.smithi159.stdout:4077 read 0x9e1773 thru 0x9e5f12 (0x47a0 bytes) 2024-09-17T22:58:08.763 INFO:teuthology.orchestra.run.smithi159.stdout:4079 write 0x6802106 thru 0x681053c (0xe437 bytes) 2024-09-17T22:58:08.766 INFO:teuthology.orchestra.run.smithi159.stdout:4080 read 0x46aae1d thru 0x46b32e9 (0x84cd bytes) 2024-09-17T22:58:08.766 INFO:teuthology.orchestra.run.smithi159.stdout:4081 read 0x2797c61 thru 0x27a2bad (0xaf4d bytes) 2024-09-17T22:58:08.766 INFO:teuthology.orchestra.run.smithi159.stdout:4082 punch from 0x3b1ebfe to 0x3b2e0d5, (0xf4d7 bytes) 2024-09-17T22:58:08.772 INFO:teuthology.orchestra.run.smithi159.stdout:4084 writesame 0x75456dd thru 0x7549a44 data_size 0xc(0x4368 bytes) 2024-09-17T22:58:08.773 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:08.773 INFO:teuthology.orchestra.run.smithi159.stdout:4089 punch from 0x5f5064 to 0x5fad39, (0x5cd5 bytes) 2024-09-17T22:58:08.773 INFO:teuthology.orchestra.run.smithi159.stdout:4090 punch from 0xcd8b1e to 0xce3d54, (0xb236 bytes) 2024-09-17T22:58:08.780 INFO:teuthology.orchestra.run.smithi159.stdout:4091 trunc from 0x7549a45 to 0xd752507 2024-09-17T22:58:08.784 INFO:teuthology.orchestra.run.smithi159.stdout:4092 write 0x8719a61 thru 0x871c31d (0x28bd bytes) 2024-09-17T22:58:08.801 INFO:teuthology.orchestra.run.smithi159.stdout:4095 trunc from 0xd752507 to 0x1bef4a2 2024-09-17T22:58:08.811 INFO:teuthology.orchestra.run.smithi159.stdout:4096 read 0x9c6503 thru 0x9d3a22 (0xd520 bytes) 2024-09-17T22:58:08.828 INFO:teuthology.orchestra.run.smithi159.stdout:4098 write 0xbad1861 thru 0xbad77a4 (0x5f44 bytes) 2024-09-17T22:58:08.830 INFO:teuthology.orchestra.run.smithi159.stdout:4099 read 0xb2d14d6 thru 0xb2de98d (0xd4b8 bytes) 2024-09-17T22:58:08.831 INFO:teuthology.orchestra.run.smithi159.stdout:4100 read 0x7476e7c thru 0x747b134 (0x42b9 bytes) 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24281 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596602131000001, 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037645445, 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24281 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:08.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:08.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232602+0000", 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232605+0000", 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-06 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232612+0000", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.596e-06 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232623+0000", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.269238+0000", 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036615385 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270084+0000", 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084551799999999996 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270247+0000", 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000162948 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.838 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24284 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596573653, 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037880098000000001, 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24284 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232633+0000", 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4320000000000002e-06 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232639+0000", 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4389999999999997e-06 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232644+0000", 2024-09-17T22:58:08.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.651e-06 2024-09-17T22:58:08.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270285+0000", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037641711000000001 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270368+0000", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2358e-05 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270510+0000", 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014204899999999999 2024-09-17T22:58:08.843 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24285 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1966080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596553491999998, 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038129107000000002, 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.844 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24285 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232652+0000", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1150000000000001e-06 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232659+0000", 2024-09-17T22:58:08.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232663+0000", 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.403e-06 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270554+0000", 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037890946000000002 2024-09-17T22:58:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270636+0000", 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1919e-05 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270779+0000", 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000143103 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24286 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596534261999999, 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038334057999999997, 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24286 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232677+0000", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1539999999999996e-06 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232683+0000", 2024-09-17T22:58:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8739999999999997e-06 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232689+0000", 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3299999999999998e-06 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270820+0000", 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038131089999999999 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270888+0000", 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8288999999999996e-05 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271003+0000", 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114955 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24289 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596507517999999, 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038485336000000002, 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24289 2024-09-17T22:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232698+0000", 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0320000000000002e-06 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232704+0000", 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232713+0000", 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2930000000000006e-06 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271036+0000", 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038323004000000001 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271078+0000", 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1063999999999999e-05 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271181+0000", 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103559 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24290 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596483273, 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038652381, 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24290 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232722+0000", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.561e-06 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232727+0000", 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6080000000000003e-06 2024-09-17T22:58:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232732+0000", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4120000000000003e-06 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271214+0000", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038481808999999999 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271268+0000", 2024-09-17T22:58:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4561999999999997e-05 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271372+0000", 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104101 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24293 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596465445, 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038796900000000002, 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24293 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232740+0000", 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232744+0000", 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232747+0000", 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.689e-06 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271401+0000", 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038653976999999999 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271442+0000", 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0547999999999999e-05 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271535+0000", 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3034000000000004e-05 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24294 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596452792000001, 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039269964999999997, 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24294 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232751+0000", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232752+0000", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1829999999999999e-06 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232755+0000", 2024-09-17T22:58:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1860000000000001e-06 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232757+0000", 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000001e-06 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271598+0000", 2024-09-17T22:58:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038840787000000002 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271794+0000", 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000195434 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272020+0000", 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022673999999999999 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24369 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1081344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596067790999999, 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039126368000000002, 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24369 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233136+0000", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2099999999999995e-07 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233141+0000", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6380000000000003e-06 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233152+0000", 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1405e-05 2024-09-17T22:58:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233160+0000", 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272065+0000", 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905397000000001 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272141+0000", 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5465999999999999e-05 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272262+0000", 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000120999 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24370 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.596030313, 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039291936999999999, 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24370 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3900000000000005e-07 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233176+0000", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233188+0000", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1535999999999999e-05 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233204+0000", 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5678000000000001e-05 2024-09-17T22:58:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272290+0000", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039085904999999997 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272356+0000", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6587999999999993e-05 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272465+0000", 2024-09-17T22:58:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108724 2024-09-17T22:58:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24373 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595989243999998, 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039461167999999998, 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24373 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233215+0000", 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233217+0000", 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:58:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233224+0000", 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9739999999999996e-06 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233230+0000", 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4809999999999997e-06 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272491+0000", 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039261457 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272560+0000", 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8576000000000002e-05 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272675+0000", 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011552699999999999 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24374 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595961184, 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039645639000000003, 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24374 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233243+0000", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233245+0000", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6000000000000001e-06 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233252+0000", 2024-09-17T22:58:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9650000000000002e-06 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233257+0000", 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2549999999999997e-06 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272704+0000", 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039446390999999997 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272778+0000", 2024-09-17T22:58:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3818000000000004e-05 2024-09-17T22:58:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272888+0000", 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110019 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24377 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1097728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595937915, 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039817298000000001, 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24377 2024-09-17T22:58:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233266+0000", 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233268+0000", 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233275+0000", 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6000000000000003e-06 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233279+0000", 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272918+0000", 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039638698 2024-09-17T22:58:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272980+0000", 2024-09-17T22:58:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1463000000000004e-05 2024-09-17T22:58:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273083+0000", 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010304799999999999 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24378 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595916715000001, 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039979973000000002, 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24378 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233289+0000", 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233295+0000", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0970000000000002e-06 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233300+0000", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7890000000000002e-06 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273107+0000", 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039806766 2024-09-17T22:58:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273167+0000", 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0541999999999997e-05 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273267+0000", 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9185999999999995e-05 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24381 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595895593000002, 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040144735000000001, 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24381 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233613+0000", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030540199999999997 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233623+0000", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.166e-06 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233630+0000", 2024-09-17T22:58:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8870000000000003e-06 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273292+0000", 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039662289000000003 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273352+0000", 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0600999999999998e-05 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273452+0000", 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9982999999999995e-05 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24382 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1228800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595560843000001, 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040027144000000001, 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24382 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233643+0000", 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233647+0000", 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8210000000000003e-06 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233654+0000", 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3259999999999998e-06 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233660+0000", 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3210000000000004e-06 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273478+0000", 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039818781999999997 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273539+0000", 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0214999999999999e-05 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273670+0000", 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013101599999999999 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24385 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595225984999999, 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039903028, 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24385 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233978+0000", 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233989+0000", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0685e-05 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234000+0000", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1282e-05 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234007+0000", 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:58:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273697+0000", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039690053000000003 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273766+0000", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8483000000000001e-05 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273880+0000", 2024-09-17T22:58:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011480300000000001 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24386 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595182736000002, 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040055687, 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24386 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234024+0000", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234035+0000", 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1319e-05 2024-09-17T22:58:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234041+0000", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8170000000000004e-06 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273907+0000", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039866255000000003 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273972+0000", 2024-09-17T22:58:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4966000000000001e-05 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274076+0000", 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104293 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24390 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1122304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.595153863, 2024-09-17T22:58:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040224370000000002, 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24390 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234050+0000", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234052+0000", 2024-09-17T22:58:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.278e-06 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234059+0000", 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234381+0000", 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032158499999999999 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274100+0000", 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039719415000000001 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274163+0000", 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3095000000000005e-05 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274274+0000", 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110373 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24391 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.594809476000002, 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040238796, 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24391 2024-09-17T22:58:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234395+0000", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1800000000000005e-07 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234398+0000", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6839999999999998e-06 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234408+0000", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-05 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234415+0000", 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274457+0000", 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040042021999999997 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274528+0000", 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1940999999999994e-05 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274633+0000", 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104114 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:08.933 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:08.933 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:08.934 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:58:08.936 INFO:teuthology.orchestra.run.smithi159.stdout:4102 trunc from 0xbad77a5 to 0x3efcc14 2024-09-17T22:58:08.936 INFO:teuthology.orchestra.run.smithi159.stdout:4105 write 0x9b127d4 thru 0x9b1bfba (0x97e7 bytes) 2024-09-17T22:58:08.936 INFO:teuthology.orchestra.run.smithi159.stdout:4106 read 0x5975543 thru 0x5978c20 (0x36de bytes) 2024-09-17T22:58:08.936 INFO:teuthology.orchestra.run.smithi159.stdout:4107 read 0x558ed65 thru 0x559d011 (0xe2ad bytes) 2024-09-17T22:58:08.936 INFO:teuthology.orchestra.run.smithi159.stdout:4110 clone 29 order 24 su 262144 sc 13 2024-09-17T22:58:08.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:08.937 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:08.932+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:08.957 INFO:teuthology.orchestra.run.smithi159.stdout:leaving image image_client.0-clone28 intact 2024-09-17T22:58:09.036 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:09.033+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:09.067 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:09.077 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T22:58:09.081 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:09.081 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:09.082 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:09.082 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:09.085 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:09.085 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:09.091 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:09.103 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:09.120 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:09.121 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:09.138 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:09.135+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:09.143 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:09.143 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:09.143 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:09.161 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:09.161 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:09.178 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:09.178 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:09.182 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:09.182 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:09.198 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.198 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:09.251 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:09.249+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:09.294 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:09.311 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.312 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.321 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:09.322 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:09.322 INFO:teuthology.orchestra.run.smithi159.stdout:4112 read 0x5f14d64 thru 0x5f16bc8 (0x1e65 bytes) 2024-09-17T22:58:09.322 INFO:teuthology.orchestra.run.smithi159.stdout:4113 clone 30 order 22 su 524288 sc 9 2024-09-17T22:58:09.340 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:09.338+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:09.395 INFO:teuthology.orchestra.run.smithi171.stderr:no valid command found; 10 closest matches: 2024-09-17T22:58:09.395 INFO:teuthology.orchestra.run.smithi171.stderr:0 2024-09-17T22:58:09.395 INFO:teuthology.orchestra.run.smithi171.stderr:1 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:2 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:abort 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:assert 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:config diff 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:config diff get 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:config get 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:config help [] 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:config set ... 2024-09-17T22:58:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:admin_socket: invalid command 2024-09-17T22:58:09.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:58:09.398 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-17T22:58:09.441 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:09.439+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:09.466 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.466 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.475 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:09.542 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:09.539+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:09.618 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.618 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:09.643 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:09.641+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:09.730 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.730 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:09.744 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:09.742+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:09.845 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:09.843+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:09.849 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.849 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.858 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:58:09.957 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:09.954+0000 7fbef5989640 -1 osd.5 45 log_to_monitors true 2024-09-17T22:58:09.959 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:09.957+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:09.967 INFO:teuthology.orchestra.run.smithi159.stdout:leaving image image_client.0-clone29 intact 2024-09-17T22:58:09.969 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:09.969 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:09.979 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:10.065 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:10.062+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:10.094 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T22:58:10.139 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.139 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.147 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:10.145+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:10.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:10.249 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:10.246+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:10.286 INFO:teuthology.orchestra.run.smithi159.stdout:4114 write 0x26f98f9 thru 0x26fa458 (0xb60 bytes) 2024-09-17T22:58:10.296 INFO:teuthology.orchestra.run.smithi159.stdout:4115 trunc from 0x9b1bfbb to 0x79b2b70 2024-09-17T22:58:10.305 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.305 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.313 INFO:teuthology.orchestra.run.smithi159.stdout:4116 writesame 0x62dd344 thru 0x62e9f06 data_size 0x1(0xcbc3 bytes) 2024-09-17T22:58:10.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:10.341 INFO:teuthology.orchestra.run.smithi159.stdout:4117 read 0x7b73b6 thru 0x7ba024 (0x2c6f bytes) 2024-09-17T22:58:10.341 INFO:teuthology.orchestra.run.smithi159.stdout:4118 write 0x76748e9 thru 0x767966d (0x4d85 bytes) 2024-09-17T22:58:10.343 INFO:teuthology.orchestra.run.smithi159.stdout:4121 write 0x42e8d75 thru 0x42f0016 (0x72a2 bytes) 2024-09-17T22:58:10.346 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.346 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.350 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:10.347+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:10.357 INFO:teuthology.orchestra.run.smithi159.stdout:4125 write 0xcd02b18 thru 0xcd0c962 (0x9e4b bytes) 2024-09-17T22:58:10.363 INFO:teuthology.orchestra.run.smithi159.stdout:4126 trunc from 0xcd0c963 to 0xe589077 2024-09-17T22:58:10.368 INFO:teuthology.orchestra.run.smithi159.stdout:4127 read 0xad8c88b thru 0xad979e2 (0xb158 bytes) 2024-09-17T22:58:10.368 INFO:teuthology.orchestra.run.smithi159.stdout:4128 write 0x1613a4f thru 0x1617123 (0x36d5 bytes) 2024-09-17T22:58:10.389 INFO:teuthology.orchestra.run.smithi159.stdout:4130 writesame 0xcfc5a07 thru 0xcfd507b data_size 0x1(0xf675 bytes) 2024-09-17T22:58:10.423 INFO:teuthology.orchestra.run.smithi159.stdout:4131 read 0x6c27041 thru 0x6c27dc0 (0xd80 bytes) 2024-09-17T22:58:10.424 INFO:teuthology.orchestra.run.smithi159.stdout:4132 punch from 0x4cdeae6 to 0x4cec51a, (0xda34 bytes) 2024-09-17T22:58:10.424 INFO:teuthology.orchestra.run.smithi159.stdout:4133 punch from 0xafe5181 to 0xafe99d9, (0x4858 bytes) 2024-09-17T22:58:10.424 INFO:teuthology.orchestra.run.smithi159.stdout:4134 write 0xe3930fb thru 0xe39f727 (0xc62d bytes) 2024-09-17T22:58:10.427 INFO:teuthology.orchestra.run.smithi159.stdout:4135 writesame 0x18ef018 thru 0x18fad78 data_size 0x1(0xbd61 bytes) 2024-09-17T22:58:10.451 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:10.448+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:10.451 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.451 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.454 INFO:teuthology.orchestra.run.smithi159.stdout:4137 read 0x20503cb thru 0x205a462 (0xa098 bytes) 2024-09-17T22:58:10.454 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.456 INFO:teuthology.orchestra.run.smithi159.stdout:4140 trunc from 0xe589077 to 0xcd3e200 2024-09-17T22:58:10.462 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:10.467 INFO:teuthology.orchestra.run.smithi159.stdout:4141 read 0x87e2f5 thru 0x88b669 (0xd375 bytes) 2024-09-17T22:58:10.470 INFO:teuthology.orchestra.run.smithi159.stdout:4142 punch from 0xc8c2528 to 0xc8d08fa, (0xe3d2 bytes) 2024-09-17T22:58:10.481 INFO:teuthology.orchestra.run.smithi159.stdout:4143 trunc from 0xcd3e200 to 0x1fea356 2024-09-17T22:58:10.552 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:10.550+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:10.580 INFO:teuthology.orchestra.run.smithi159.stdout:4144 write 0xe76e9f4 thru 0xe779ef4 (0xb501 bytes) 2024-09-17T22:58:10.583 INFO:teuthology.orchestra.run.smithi159.stdout:4146 writesame 0x90c5228 thru 0x90d1f45 data_size 0x3b(0xcd1e bytes) 2024-09-17T22:58:10.586 INFO:teuthology.orchestra.run.smithi159.stdout:4147 writesame 0xa5a606b thru 0xa5b1148 data_size 0x1(0xb0de bytes) 2024-09-17T22:58:10.601 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.601 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.610 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:10.610 INFO:teuthology.orchestra.run.smithi159.stdout:4149 write 0x73a7f81 thru 0x73aeab8 (0x6b38 bytes) 2024-09-17T22:58:10.612 INFO:teuthology.orchestra.run.smithi159.stdout:4150 read 0x1b6b682 thru 0x1b6b8b8 (0x237 bytes) 2024-09-17T22:58:10.613 INFO:teuthology.orchestra.run.smithi159.stdout:4151 write 0xb1b2840 thru 0xb1b84e2 (0x5ca3 bytes) 2024-09-17T22:58:10.616 INFO:teuthology.orchestra.run.smithi159.stdout:4152 write 0x12be5fd thru 0x12c9554 (0xaf58 bytes) 2024-09-17T22:58:10.618 INFO:teuthology.orchestra.run.smithi159.stdout:4153 punch from 0x3c51bf9 to 0x3c5d0d1, (0xb4d8 bytes) 2024-09-17T22:58:10.618 INFO:teuthology.orchestra.run.smithi159.stdout:4155 read 0xc6b29cb thru 0xc6bd252 (0xa888 bytes) 2024-09-17T22:58:10.618 INFO:teuthology.orchestra.run.smithi159.stdout:4156 write 0x39b8e40 thru 0x39c6f8a (0xe14b bytes) 2024-09-17T22:58:10.621 INFO:teuthology.orchestra.run.smithi159.stdout:4157 write 0x251c1dc thru 0x2522837 (0x665c bytes) 2024-09-17T22:58:10.624 INFO:teuthology.orchestra.run.smithi159.stdout:4158 read 0x3c81cf0 thru 0x3c89ffb (0x830c bytes) 2024-09-17T22:58:10.624 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.624 INFO:teuthology.orchestra.run.smithi159.stdout:4160 write 0xae107b thru 0xae7fa3 (0x6f29 bytes) 2024-09-17T22:58:10.625 INFO:teuthology.orchestra.run.smithi159.stdout:4161 writesame 0xcc1e708 thru 0xcc21b9d data_size 0x2(0x3496 bytes) 2024-09-17T22:58:10.633 INFO:teuthology.orchestra.run.smithi159.stdout:4162 trunc from 0xe779ef5 to 0xc120da1 2024-09-17T22:58:10.645 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.645 INFO:teuthology.orchestra.run.smithi159.stdout:4165 write 0x7b1d339 thru 0x7b22e99 (0x5b61 bytes) 2024-09-17T22:58:10.648 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.653 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:10.651+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:10.655 INFO:teuthology.orchestra.run.smithi159.stdout:4168 write 0xd8cd221 thru 0xd8da824 (0xd604 bytes) 2024-09-17T22:58:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:4169 punch from 0x9be66a1 to 0x9bebac7, (0x5426 bytes) 2024-09-17T22:58:10.665 INFO:teuthology.orchestra.run.smithi159.stdout:4172 trunc from 0xd8da825 to 0x81ec98b 2024-09-17T22:58:10.688 INFO:teuthology.orchestra.run.smithi159.stdout:4174 writesame 0xd5d05e4 thru 0xd5d32a5 data_size 0x11(0x2cc2 bytes) 2024-09-17T22:58:10.688 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.688 INFO:teuthology.orchestra.run.smithi159.stdout:4176 punch from 0x4ea38d to 0x4ef128, (0x4d9b bytes) 2024-09-17T22:58:10.689 INFO:teuthology.orchestra.run.smithi159.stdout:4177 read 0x94119f thru 0x94ac15 (0x9a77 bytes) 2024-09-17T22:58:10.689 INFO:teuthology.orchestra.run.smithi159.stdout:4178 write 0xca1159f thru 0xca1f256 (0xdcb8 bytes) 2024-09-17T22:58:10.692 INFO:teuthology.orchestra.run.smithi159.stdout:4179 punch from 0x492ec98 to 0x4933ae8, (0x4e50 bytes) 2024-09-17T22:58:10.692 INFO:teuthology.orchestra.run.smithi159.stdout:4180 writesame 0x1f82d68 thru 0x1f8968f data_size 0x14(0x6928 bytes) 2024-09-17T22:58:10.694 INFO:teuthology.orchestra.run.smithi159.stdout:4181 write 0xd4a9a80 thru 0xd4b3722 (0x9ca3 bytes) 2024-09-17T22:58:10.697 INFO:teuthology.orchestra.run.smithi159.stdout:4182 read 0xb5d5863 thru 0xb5e19fb (0xc199 bytes) 2024-09-17T22:58:10.697 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.697 INFO:teuthology.orchestra.run.smithi159.stdout:4184 write 0x4537f2b thru 0x45463bd (0xe493 bytes) 2024-09-17T22:58:10.700 INFO:teuthology.orchestra.run.smithi159.stdout:4186 write 0xae13169 thru 0xae1ecf1 (0xbb89 bytes) 2024-09-17T22:58:10.703 INFO:teuthology.orchestra.run.smithi159.stdout:4187 read 0xaa0f21f thru 0xaa1d765 (0xe547 bytes) 2024-09-17T22:58:10.703 INFO:teuthology.orchestra.run.smithi159.stdout:4189 write 0x4e07162 thru 0x4e0eb5f (0x79fe bytes) 2024-09-17T22:58:10.706 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.707 INFO:teuthology.orchestra.run.smithi159.stdout:4191 trunc from 0xd5d32a6 to 0xc420054 2024-09-17T22:58:10.714 INFO:teuthology.orchestra.run.smithi159.stdout:4192 punch from 0xbd7b65b to 0xbd7ef24, (0x38c9 bytes) 2024-09-17T22:58:10.714 INFO:teuthology.orchestra.run.smithi159.stdout:4194 writesame 0xb708fb4 thru 0xb710976 data_size 0x7(0x79c3 bytes) 2024-09-17T22:58:10.719 INFO:teuthology.orchestra.run.smithi159.stdout:4195 write 0x370fa72 thru 0x3713414 (0x39a3 bytes) 2024-09-17T22:58:10.732 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.733 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.734 INFO:teuthology.orchestra.run.smithi159.stdout:4196 trunc from 0xc420054 to 0xc16004 2024-09-17T22:58:10.746 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:10.755 INFO:teuthology.orchestra.run.smithi159.stdout:4197 read 0x10d39f thru 0x116550 (0x91b2 bytes) 2024-09-17T22:58:10.756 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:10.753+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:10.756 INFO:teuthology.orchestra.run.smithi159.stdout:4198 punch from 0x7cadd1 to 0x7cb314, (0x543 bytes) 2024-09-17T22:58:10.756 INFO:teuthology.orchestra.run.smithi159.stdout:4199 read 0xbf539f thru 0xbfa075 (0x4cd7 bytes) 2024-09-17T22:58:10.776 INFO:teuthology.orchestra.run.smithi159.stdout:4200 writesame 0xc33c533 thru 0xc3470bf data_size 0x1(0xab8d bytes) 2024-09-17T22:58:10.776 INFO:teuthology.orchestra.run.smithi159.stdout:4201 write 0x65608c5 thru 0x656b32d (0xaa69 bytes) 2024-09-17T22:58:10.779 INFO:teuthology.orchestra.run.smithi159.stdout:4202 writesame 0x9f16d05 thru 0x9f1ceb8 data_size 0x34(0x61b4 bytes) 2024-09-17T22:58:10.782 INFO:teuthology.orchestra.run.smithi159.stdout:4204 read 0x64b04c9 thru 0x64bfbc9 (0xf701 bytes) 2024-09-17T22:58:10.796 INFO:teuthology.orchestra.run.smithi159.stdout:4208 trunc from 0xc3470c0 to 0xec07a7 2024-09-17T22:58:10.808 INFO:teuthology.orchestra.run.smithi159.stdout:4209 read 0x66c9b9 thru 0x66d862 (0xeaa bytes) 2024-09-17T22:58:10.821 INFO:teuthology.orchestra.run.smithi159.stdout:4211 write 0x6231f88 thru 0x62393f9 (0x7472 bytes) 2024-09-17T22:58:10.827 INFO:teuthology.orchestra.run.smithi159.stdout:4212 trunc from 0x62393fa to 0x83c7269 2024-09-17T22:58:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:4214 write 0x4f36b89 thru 0x4f41dce (0xb246 bytes) 2024-09-17T22:58:10.835 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:10.835 INFO:teuthology.orchestra.run.smithi159.stdout:4218 read 0x46fb525 thru 0x46fd690 (0x216c bytes) 2024-09-17T22:58:10.835 INFO:teuthology.orchestra.run.smithi159.stdout:4220 punch from 0x67664d6 to 0x676f977, (0x94a1 bytes) 2024-09-17T22:58:10.835 INFO:teuthology.orchestra.run.smithi159.stdout:4222 write 0x1c999cd thru 0x1ca0002 (0x6636 bytes) 2024-09-17T22:58:10.836 INFO:teuthology.orchestra.run.smithi159.stdout:4223 write 0x478f467 thru 0x4794d5a (0x58f4 bytes) 2024-09-17T22:58:10.839 INFO:teuthology.orchestra.run.smithi159.stdout:4225 clone 31 order 24 su 262144 sc 10 2024-09-17T22:58:10.856 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:10.854+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:10.873 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:10.874 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:10.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:10.936 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:10.934+0000 7fbee9738640 -1 osd.5 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:58:10.957 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:10.955+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:10.981 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone30 from 0x83c7269 (overlap 0xc16004) to 0xae099d 2024-09-17T22:58:11.040 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:11.040 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:11.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:11.073 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T22:58:11.073 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:11.071+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:11.159 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:11.156+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:11.178 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:11.178 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:11.183 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:11.188 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:11.271 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:11.269+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:11.311 INFO:teuthology.orchestra.run.smithi159.stdout:4226 write 0xb80582d thru 0xb814d63 (0xf537 bytes) 2024-09-17T22:58:11.312 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:11.312 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:11.314 INFO:teuthology.orchestra.run.smithi159.stdout:4227 write 0x38c9d6f thru 0x38d77ae (0xda40 bytes) 2024-09-17T22:58:11.324 INFO:teuthology.orchestra.run.smithi159.stdout:4228 writesame 0x44100b7 thru 0x4419b41 data_size 0x1(0x9a8b bytes) 2024-09-17T22:58:11.333 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:11.333 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:11.345 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:11.345 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:11.349 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:11.349 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:11.353 INFO:teuthology.orchestra.run.smithi159.stdout:4229 trunc from 0xb814d64 to 0xd40eeb7 2024-09-17T22:58:11.354 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:11.362 INFO:teuthology.orchestra.run.smithi159.stdout:4230 trunc from 0xd40eeb7 to 0xa9a609d 2024-09-17T22:58:11.362 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:11.360+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:11.369 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.375 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:11.375 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:11.375 INFO:teuthology.orchestra.run.smithi159.stdout:4232 trunc from 0xa9a609d to 0x57ee0bc 2024-09-17T22:58:11.402 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:11.403 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:11.403 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T22:58:11.422 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:11.422 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:11.425 INFO:teuthology.orchestra.run.smithi159.stdout:4233 writesame 0xa236417 thru 0xa237eaa data_size 0x2(0x1a94 bytes) 2024-09-17T22:58:11.431 INFO:teuthology.orchestra.run.smithi159.stdout:4235 write 0xae69b08 thru 0xae6af9c (0x1495 bytes) 2024-09-17T22:58:11.434 INFO:teuthology.orchestra.run.smithi159.stdout:4236 writesame 0xa208950 thru 0xa20a303 data_size 0x14(0x19b4 bytes) 2024-09-17T22:58:11.438 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.438 INFO:teuthology.orchestra.run.smithi159.stdout:4239 writesame 0x8c6729d thru 0x8c6f792 data_size 0x6(0x84f6 bytes) 2024-09-17T22:58:11.442 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:11.442 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:11.446 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:11.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:11.449 INFO:teuthology.orchestra.run.smithi159.stdout:4240 write 0xd080bb9 thru 0xd08b0fb (0xa543 bytes) 2024-09-17T22:58:11.452 INFO:teuthology.orchestra.run.smithi159.stdout:4241 write 0xcc405ee thru 0xcc4e748 (0xe15b bytes) 2024-09-17T22:58:11.454 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.454 INFO:teuthology.orchestra.run.smithi159.stdout:4244 write 0xa516cc3 thru 0xa51901e (0x235c bytes) 2024-09-17T22:58:11.457 INFO:teuthology.orchestra.run.smithi159.stdout:4247 punch from 0x8c7e295 to 0x8c84067, (0x5dd2 bytes) 2024-09-17T22:58:11.458 INFO:teuthology.orchestra.run.smithi159.stdout:4248 read 0x56c8fcc thru 0x56d7528 (0xe55d bytes) 2024-09-17T22:58:11.458 INFO:teuthology.orchestra.run.smithi159.stdout:4249 write 0x402c46c thru 0x403a1fc (0xdd91 bytes) 2024-09-17T22:58:11.460 INFO:teuthology.orchestra.run.smithi159.stdout:4250 write 0xb031071 thru 0xb03d3dd (0xc36d bytes) 2024-09-17T22:58:11.461 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:11.458+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:11.461 INFO:teuthology.orchestra.run.smithi159.stdout:4251 write 0xc808407 thru 0xc80b1d5 (0x2dcf bytes) 2024-09-17T22:58:11.462 INFO:teuthology.orchestra.run.smithi159.stdout:4253 write 0x6d47bdf thru 0x6d564f4 (0xe916 bytes) 2024-09-17T22:58:11.463 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.464 INFO:teuthology.orchestra.run.smithi159.stdout:4255 writesame 0x4e0982d thru 0x4e13bba data_size 0x6a(0xa38e bytes) 2024-09-17T22:58:11.469 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.470 INFO:teuthology.orchestra.run.smithi159.stdout:4257 write 0xbdb5153 thru 0xbdbcc23 (0x7ad1 bytes) 2024-09-17T22:58:11.470 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.471 INFO:teuthology.orchestra.run.smithi159.stdout:4259 trunc from 0xd08b0fc to 0x9503350 2024-09-17T22:58:11.482 INFO:teuthology.orchestra.run.smithi159.stdout:4260 read 0x7767f4c thru 0x776d10e (0x51c3 bytes) 2024-09-17T22:58:11.491 INFO:teuthology.orchestra.run.smithi159.stdout:4262 trunc from 0x9503350 to 0x3493174 2024-09-17T22:58:11.494 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5757 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904405715999999, 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038199444999999999, 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5757 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:11.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586462+0000", 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5600000000000002e-07 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586464+0000", 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.137e-06 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586468+0000", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8630000000000001e-06 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586471+0000", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7549999999999999e-06 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624560+0000", 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038088960999999998 2024-09-17T22:58:11.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624607+0000", 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6740000000000003e-05 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624661+0000", 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4732999999999999e-05 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.499 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5758 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904392827000002, 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038315071999999999, 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5758 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586475+0000", 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5100000000000001e-07 2024-09-17T22:58:11.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586476+0000", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.113e-06 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586483+0000", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5200000000000003e-06 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586487+0000", 2024-09-17T22:58:11.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3479999999999997e-06 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624681+0000", 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038194371999999997 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624722+0000", 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0367999999999997e-05 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624790+0000", 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8100000000000002e-05 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5759 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3948544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904377101999998, 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038450864000000001, 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5759 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586490+0000", 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586491+0000", 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586492+0000", 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5349999999999999e-06 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586495+0000", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586498+0000", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6170000000000001e-06 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624828+0000", 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038329514000000002 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624875+0000", 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7107999999999997e-05 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624941+0000", 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6647000000000001e-05 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.508 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5760 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904366394999997, 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038669935000000003, 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5760 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586501+0000", 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:58:11.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586503+0000", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0899999999999999e-06 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586506+0000", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586508+0000", 2024-09-17T22:58:11.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.624980+0000", 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038472343999999999 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625076+0000", 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5214e-05 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625171+0000", 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5421999999999995e-05 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5761 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4276224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904356415000002, 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038805352000000001, 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5761 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:11.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586511+0000", 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.22e-07 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586512+0000", 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0210000000000001e-06 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586515+0000", 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586518+0000", 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625211+0000", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038693268000000003 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625261+0000", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9345000000000003e-05 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625317+0000", 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5850999999999998e-05 2024-09-17T22:58:11.519 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5762 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4472832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904346523999997, 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038916981000000003, 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5762 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586521+0000", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586525+0000", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.224e-06 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586532+0000", 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3950000000000003e-06 2024-09-17T22:58:11.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586535+0000", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625339+0000", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038804584000000003 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625385+0000", 2024-09-17T22:58:11.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5099000000000003e-05 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625438+0000", 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3511999999999999e-05 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5763 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4669440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:11.524 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904328593999999, 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039029253999999999, 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5763 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586539+0000", 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586541+0000", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.268e-06 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586547+0000", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586554+0000", 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625465+0000", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038910924 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625511+0000", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5179999999999998e-05 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625568+0000", 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7738999999999997e-05 2024-09-17T22:58:11.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5764 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 4866048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904307742999997, 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039122912000000003, 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5764 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:11.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586560+0000", 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586561+0000", 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.266e-06 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586566+0000", 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586568+0000", 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625592+0000", 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039024174000000002 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625634+0000", 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1464000000000001e-05 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625683+0000", 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9002000000000003e-05 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.533 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5765 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5062656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.904295896000001, 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039240865999999999, 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5765 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586572+0000", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586895+0000", 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000323658 2024-09-17T22:58:11.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586899+0000", 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6260000000000001e-06 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586910+0000", 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1239e-05 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586917+0000", 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625710+0000", 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038792608999999999 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625757+0000", 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7416000000000001e-05 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625813+0000", 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5395000000000001e-05 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5766 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5259264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.903939764, 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038992026999999999, 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.538 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5766 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586928+0000", 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586931+0000", 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586939+0000", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2549999999999997e-06 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586945+0000", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625828+0000", 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038882947000000001 2024-09-17T22:58:11.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625871+0000", 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3204000000000002e-05 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625920+0000", 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8539999999999999e-05 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5767 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5455872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.903914673999999, 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.039097260000000002, 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5767 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.586953+0000", 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588020+0000", 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00106675 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588024+0000", 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588033+0000", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588044+0000", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1800000000000001e-05 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625943+0000", 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.037898695000000003 2024-09-17T22:58:11.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.625988+0000", 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4577999999999999e-05 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626050+0000", 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2605000000000001e-05 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5769 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 5849088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.902784042, 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038207729000000003, 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5769 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.548 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588084+0000", 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588086+0000", 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.103e-06 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588094+0000", 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6459999999999993e-06 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588099+0000", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1259999999999997e-06 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626190+0000", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.03809067 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626234+0000", 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4051999999999998e-05 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626291+0000", 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7694e-05 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5770 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.552 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.902761069999997, 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038303865999999999, 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5770 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588107+0000", 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588109+0000", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0899999999999999e-06 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588116+0000", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2439999999999996e-06 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588122+0000", 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8640000000000001e-06 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626315+0000", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038193105999999998 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626359+0000", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3828000000000001e-05 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626410+0000", 2024-09-17T22:58:11.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1328000000000001e-05 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5771 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.902738790000001, 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038402700999999997, 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5771 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588129+0000", 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588131+0000", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588138+0000", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3049999999999998e-06 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588143+0000", 2024-09-17T22:58:11.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4650000000000002e-06 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626433+0000", 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038289895999999997 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626476+0000", 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2913000000000001e-05 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626532+0000", 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5692000000000001e-05 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5772 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 6438912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.902717131999999, 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038494234000000002, 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.562 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5772 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.588150+0000", 2024-09-17T22:58:11.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590017+0000", 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018664980000000001 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590020+0000", 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4769999999999999e-06 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590027+0000", 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5209999999999999e-06 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590032+0000", 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0810000000000001e-06 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626554+0000", 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036521551999999999 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626595+0000", 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1900000000000002e-05 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.626645+0000", 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9205000000000001e-05 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5829 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.566 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.900512669999998, 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038295726000000002, 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5829 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.590355+0000", 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592028+0000", 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001673379 2024-09-17T22:58:11.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592032+0000", 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5109999999999999e-06 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592038+0000", 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7379999999999999e-06 2024-09-17T22:58:11.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592042+0000", 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628543+0000", 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036500828999999999 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628594+0000", 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1737999999999998e-05 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.628651+0000", 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6320999999999998e-05 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5840 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.571 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.898662035000001, 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038232347, 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5840 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592206+0000", 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592208+0000", 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-06 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592214+0000", 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7479999999999999e-06 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592220+0000", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.037e-06 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630310+0000", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038090474999999999 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630373+0000", 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2467999999999999e-05 2024-09-17T22:58:11.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630438+0000", 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5235999999999997e-05 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5841 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3375104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.898641797000003, 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038361579999999999, 2024-09-17T22:58:11.576 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5841 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592226+0000", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592228+0000", 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:58:11.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592235+0000", 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8040000000000004e-06 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592242+0000", 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2230000000000001e-06 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630468+0000", 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038225982999999998 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630517+0000", 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8656000000000003e-05 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630587+0000", 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0698000000000006e-05 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:5842 3.1e 3:7eca3d69:::rbd_data.1188b3deeabb.0000000000000002:8 [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 40.898619066999998, 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.038469774999999998, 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 5842 2024-09-17T22:58:11.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.592249+0000", 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594021+0000", 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001772849 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594026+0000", 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4399999999999998e-06 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594037+0000", 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1071e-05 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.594044+0000", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9129999999999997e-06 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630613+0000", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.036569602999999999 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630659+0000", 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5246e-05 2024-09-17T22:58:11.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:30.630718+0000", 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9653e-05 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 12.337325926, 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:11.585 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:11.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:11.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:11.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:11.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:11.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:11.592 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:11.594 INFO:teuthology.orchestra.run.smithi159.stdout:4263 read 0xc73e48 thru 0xc8063a (0xc7f3 bytes) 2024-09-17T22:58:11.594 INFO:teuthology.orchestra.run.smithi159.stdout:4264 read 0x279e0ba thru 0x27a6fde (0x8f25 bytes) 2024-09-17T22:58:11.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:4266 writesame 0x9a48750 thru 0x9a54b24 data_size 0x3(0xc3d5 bytes) 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:4267 read 0x269d842 thru 0x26a7580 (0x9d3f bytes) 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:4268 write 0x7ce1c54 thru 0x7ce847e (0x682b bytes) 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:4271 write 0x9da2e79 thru 0x9da6f15 (0x409d bytes) 2024-09-17T22:58:11.596 INFO:teuthology.orchestra.run.smithi159.stdout:4272 clone 32 order 22 su 131072 sc 5 2024-09-17T22:58:11.597 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:11.591+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:11.695 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:11.692+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:11.698 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:11.699 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:11.699 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:11.707 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:11.796 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:11.793+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:11.811 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:11.811 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:11.811 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:11.819 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:11.897 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:11.894+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:11.918 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24281 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696917477, 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037645445, 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24281 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232601+0000", 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232602+0000", 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232605+0000", 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-06 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232612+0000", 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.596e-06 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232623+0000", 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1093e-05 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.269238+0000", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.036615385 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270084+0000", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00084551799999999996 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270247+0000", 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000162948 2024-09-17T22:58:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24284 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696888998999999, 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037880098000000001, 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24284 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232630+0000", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232633+0000", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4320000000000002e-06 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232639+0000", 2024-09-17T22:58:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4389999999999997e-06 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232644+0000", 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.651e-06 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270285+0000", 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037641711000000001 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270368+0000", 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2358e-05 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270510+0000", 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014204899999999999 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24285 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1966080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696868838, 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038129107000000002, 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24285 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232650+0000", 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232652+0000", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1150000000000001e-06 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232659+0000", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232663+0000", 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.403e-06 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270554+0000", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.037890946000000002 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270636+0000", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1919e-05 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270779+0000", 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000143103 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24286 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696849608000001, 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038334057999999997, 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24286 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232669+0000", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232677+0000", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1539999999999996e-06 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232683+0000", 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8739999999999997e-06 2024-09-17T22:58:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232689+0000", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3299999999999998e-06 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270820+0000", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038131089999999999 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.270888+0000", 2024-09-17T22:58:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8288999999999996e-05 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271003+0000", 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114955 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24289 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696822864000001, 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038485336000000002, 2024-09-17T22:58:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24289 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232696+0000", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232698+0000", 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0320000000000002e-06 2024-09-17T22:58:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232704+0000", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8610000000000003e-06 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232713+0000", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2930000000000006e-06 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271036+0000", 2024-09-17T22:58:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038323004000000001 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271078+0000", 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1063999999999999e-05 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271181+0000", 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103559 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24290 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1974272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696798618999999, 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038652381, 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24290 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232720+0000", 2024-09-17T22:58:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232722+0000", 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.561e-06 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232727+0000", 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6080000000000003e-06 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232732+0000", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4120000000000003e-06 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271214+0000", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038481808999999999 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271268+0000", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4561999999999997e-05 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271372+0000", 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104101 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24293 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696780790999998, 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038796900000000002, 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24293 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232738+0000", 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232740+0000", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.074e-06 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232744+0000", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232747+0000", 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.689e-06 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271401+0000", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038653976999999999 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271442+0000", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0547999999999999e-05 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271535+0000", 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3034000000000004e-05 2024-09-17T22:58:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24294 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696768137999999, 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039269964999999997, 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24294 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232750+0000", 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232751+0000", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232752+0000", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1829999999999999e-06 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232755+0000", 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1860000000000001e-06 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.232757+0000", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000001e-06 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271598+0000", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038840787000000002 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.271794+0000", 2024-09-17T22:58:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000195434 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272020+0000", 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022673999999999999 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24369 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1081344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696383137000002, 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039126368000000002, 2024-09-17T22:58:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24369 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233135+0000", 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233136+0000", 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2099999999999995e-07 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233141+0000", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6380000000000003e-06 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233152+0000", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1405e-05 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233160+0000", 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272065+0000", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.038905397000000001 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272141+0000", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5465999999999999e-05 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272262+0000", 2024-09-17T22:58:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000120999 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24370 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696345658999999, 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039291936999999999, 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24370 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233173+0000", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3900000000000005e-07 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233176+0000", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233188+0000", 2024-09-17T22:58:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1535999999999999e-05 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233204+0000", 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5678000000000001e-05 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272290+0000", 2024-09-17T22:58:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039085904999999997 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272356+0000", 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6587999999999993e-05 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272465+0000", 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108724 2024-09-17T22:58:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24373 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.69630459, 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039461167999999998, 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24373 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233214+0000", 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233215+0000", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2799999999999996e-07 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233217+0000", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233224+0000", 2024-09-17T22:58:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9739999999999996e-06 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233230+0000", 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4809999999999997e-06 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272491+0000", 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039261457 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272560+0000", 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8576000000000002e-05 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272675+0000", 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011552699999999999 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24374 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696276529999999, 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039645639000000003, 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24374 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233242+0000", 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233243+0000", 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9100000000000004e-07 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233245+0000", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6000000000000001e-06 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233252+0000", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9650000000000002e-06 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233257+0000", 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2549999999999997e-06 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272704+0000", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039446390999999997 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272778+0000", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3818000000000004e-05 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272888+0000", 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110019 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24377 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1097728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696253260999999, 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039817298000000001, 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24377 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233265+0000", 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233266+0000", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233268+0000", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233275+0000", 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6000000000000003e-06 2024-09-17T22:58:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233279+0000", 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8690000000000003e-06 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272918+0000", 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039638698 2024-09-17T22:58:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.272980+0000", 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1463000000000004e-05 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273083+0000", 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010304799999999999 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24378 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696232061, 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039979973000000002, 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24378 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233287+0000", 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.89e-07 2024-09-17T22:58:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233289+0000", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233295+0000", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0970000000000002e-06 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233300+0000", 2024-09-17T22:58:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7890000000000002e-06 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273107+0000", 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039806766 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273167+0000", 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0541999999999997e-05 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273267+0000", 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9185999999999995e-05 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24381 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.696210939, 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040144735000000001, 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24381 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233308+0000", 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233613+0000", 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00030540199999999997 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233623+0000", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.166e-06 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233630+0000", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8870000000000003e-06 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273292+0000", 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039662289000000003 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273352+0000", 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0600999999999998e-05 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273452+0000", 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9982999999999995e-05 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24382 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1228800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.695876189, 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040027144000000001, 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24382 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233642+0000", 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233643+0000", 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233647+0000", 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8210000000000003e-06 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233654+0000", 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3259999999999998e-06 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233660+0000", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3210000000000004e-06 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273478+0000", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039818781999999997 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273539+0000", 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0214999999999999e-05 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273670+0000", 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013101599999999999 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24385 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.695541331000001, 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039903028, 2024-09-17T22:58:12.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24385 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233977+0000", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233978+0000", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.233989+0000", 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0685e-05 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234000+0000", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1282e-05 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234007+0000", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273697+0000", 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039690053000000003 2024-09-17T22:58:12.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273766+0000", 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8483000000000001e-05 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273880+0000", 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011480300000000001 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24386 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.695498082, 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040055687, 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24386 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234021+0000", 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:58:12.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234024+0000", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234035+0000", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1319e-05 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234041+0000", 2024-09-17T22:58:12.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8170000000000004e-06 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273907+0000", 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039866255000000003 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.273972+0000", 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4966000000000001e-05 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274076+0000", 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104293 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24390 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1122304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.695469208999999, 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040224370000000002, 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.015 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24390 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234049+0000", 2024-09-17T22:58:12.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234050+0000", 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234052+0000", 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.278e-06 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234059+0000", 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234381+0000", 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00032158499999999999 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274100+0000", 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.039719415000000001 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274163+0000", 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3095000000000005e-05 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274274+0000", 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000110373 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:24391 3.1c 3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:10 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T22:58:12.019 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 25.695124822, 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040238796, 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 24391 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234394+0000", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234395+0000", 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1800000000000005e-07 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234398+0000", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6839999999999998e-06 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234408+0000", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.026e-05 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.234415+0000", 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0569999999999998e-06 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274457+0000", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.040042021999999997 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274528+0000", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1940999999999994e-05 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:46.274633+0000", 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104114 2024-09-17T22:58:12.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.024 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.025 INFO:teuthology.orchestra.run.smithi159.stdout:leaving image image_client.0-clone31 intact 2024-09-17T22:58:12.025 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:12.026 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:12.023+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:12.126 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:12.124+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:12.175 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T22:58:12.177 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:12.177 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:12.177 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.187 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:12.197 INFO:teuthology.orchestra.run.smithi159.stdout:4273 punch from 0x52370ec to 0x52430b8, (0xbfcc bytes) 2024-09-17T22:58:12.212 INFO:teuthology.orchestra.run.smithi159.stdout:4278 trunc from 0x9da6f16 to 0x89847b 2024-09-17T22:58:12.229 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:12.226+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:12.264 INFO:teuthology.orchestra.run.smithi159.stdout:4279 writesame 0xd1e28d3 thru 0xd1e9eec data_size 0x3(0x761a bytes) 2024-09-17T22:58:12.264 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.264 INFO:teuthology.orchestra.run.smithi159.stdout:4281 writesame 0x8f3531 thru 0x8f395a data_size 0xd(0x42a bytes) 2024-09-17T22:58:12.265 INFO:teuthology.orchestra.run.smithi159.stdout:4282 write 0x1d8cb9c thru 0x1d96125 (0x958a bytes) 2024-09-17T22:58:12.273 INFO:teuthology.orchestra.run.smithi159.stdout:4287 trunc from 0xd1e9eed to 0x88c2e36 2024-09-17T22:58:12.278 INFO:teuthology.orchestra.run.smithi159.stdout:4289 punch from 0x639e337 to 0x63a0781, (0x244a bytes) 2024-09-17T22:58:12.278 INFO:teuthology.orchestra.run.smithi159.stdout:4291 read 0x719809a thru 0x71a033c (0x82a3 bytes) 2024-09-17T22:58:12.278 INFO:teuthology.orchestra.run.smithi159.stdout:4292 writesame 0x5f780bd thru 0x5f7c3fd data_size 0x3(0x4341 bytes) 2024-09-17T22:58:12.282 INFO:teuthology.orchestra.run.smithi159.stdout:4293 read 0x70af51a thru 0x70bded5 (0xe9bc bytes) 2024-09-17T22:58:12.292 INFO:teuthology.orchestra.run.smithi159.stdout:4294 writesame 0xa372f53 thru 0xa37313b data_size 0x1(0x1e9 bytes) 2024-09-17T22:58:12.292 INFO:teuthology.orchestra.run.smithi159.stdout:4295 read 0x798dfda thru 0x79935f4 (0x561b bytes) 2024-09-17T22:58:12.294 INFO:teuthology.orchestra.run.smithi159.stdout:4297 writesame 0xa98a0e1 thru 0xa9971db data_size 0x11(0xd0fb bytes) 2024-09-17T22:58:12.295 INFO:teuthology.orchestra.run.smithi159.stdout:4298 read 0x681cca1 thru 0x681f848 (0x2ba8 bytes) 2024-09-17T22:58:12.295 INFO:teuthology.orchestra.run.smithi159.stdout:4299 write 0x29f87c1 thru 0x2a05220 (0xca60 bytes) 2024-09-17T22:58:12.298 INFO:teuthology.orchestra.run.smithi159.stdout:4300 read 0x7264bf1 thru 0x726dc9e (0x90ae bytes) 2024-09-17T22:58:12.298 INFO:teuthology.orchestra.run.smithi159.stdout:4301 write 0x6e071d8 thru 0x6e075e8 (0x411 bytes) 2024-09-17T22:58:12.299 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:12.299 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:12.299 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.301 INFO:teuthology.orchestra.run.smithi159.stdout:4302 punch from 0x88c3ee5 to 0x88c842c, (0x4547 bytes) 2024-09-17T22:58:12.301 INFO:teuthology.orchestra.run.smithi159.stdout:4303 punch from 0x4f8ba82 to 0x4f993ec, (0xd96a bytes) 2024-09-17T22:58:12.301 INFO:teuthology.orchestra.run.smithi159.stdout:4304 punch from 0x4b00367 to 0x4b0798a, (0x7623 bytes) 2024-09-17T22:58:12.301 INFO:teuthology.orchestra.run.smithi159.stdout:4305 punch from 0x5744e42 to 0x5749342, (0x4500 bytes) 2024-09-17T22:58:12.301 INFO:teuthology.orchestra.run.smithi159.stdout:4306 read 0x942e208 thru 0x94314e3 (0x32dc bytes) 2024-09-17T22:58:12.302 INFO:teuthology.orchestra.run.smithi159.stdout:4307 read 0x9cc062a thru 0x9ccd058 (0xca2f bytes) 2024-09-17T22:58:12.302 INFO:teuthology.orchestra.run.smithi159.stdout:4308 write 0x5ff5bbf thru 0x5ff7a24 (0x1e66 bytes) 2024-09-17T22:58:12.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:12.309 INFO:teuthology.orchestra.run.smithi159.stdout:4310 write 0xb0b2ac3 thru 0xb0b4d52 (0x2290 bytes) 2024-09-17T22:58:12.322 INFO:teuthology.orchestra.run.smithi159.stdout:4312 trunc from 0xb0b4d53 to 0x20fc8e1 2024-09-17T22:58:12.341 INFO:teuthology.orchestra.run.smithi159.stdout:4313 trunc from 0x20fc8e1 to 0x905a82e 2024-09-17T22:58:12.346 INFO:teuthology.orchestra.run.smithi159.stdout:4314 read 0x79bf835 thru 0x79c7956 (0x8122 bytes) 2024-09-17T22:58:12.346 INFO:teuthology.orchestra.run.smithi159.stdout:4315 read 0x43a259 thru 0x447c7e (0xda26 bytes) 2024-09-17T22:58:12.347 INFO:teuthology.orchestra.run.smithi159.stdout:4316 read 0x82857b6 thru 0x8286a7d (0x12c8 bytes) 2024-09-17T22:58:12.352 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:12.350+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:12.354 INFO:teuthology.orchestra.run.smithi159.stdout:4317 trunc from 0x905a82e to 0x347b0ca 2024-09-17T22:58:12.363 INFO:teuthology.orchestra.run.smithi159.stdout:4320 trunc from 0x347b0ca to 0x72b540 2024-09-17T22:58:12.380 INFO:teuthology.orchestra.run.smithi159.stdout:4321 trunc from 0x72b540 to 0x33e929f 2024-09-17T22:58:12.385 INFO:teuthology.orchestra.run.smithi159.stdout:4322 read 0x30e2c33 thru 0x30f0cd1 (0xe09f bytes) 2024-09-17T22:58:12.385 INFO:teuthology.orchestra.run.smithi159.stdout:4324 read 0x1041bf1 thru 0x10433ee (0x17fe bytes) 2024-09-17T22:58:12.404 INFO:teuthology.orchestra.run.smithi159.stdout:4325 write 0xd8cdf45 thru 0xd8d07c0 (0x287c bytes) 2024-09-17T22:58:12.406 INFO:teuthology.orchestra.run.smithi159.stdout:4326 writesame 0x8f7fe09 thru 0x8f8d6b4 data_size 0x62(0xd8ac bytes) 2024-09-17T22:58:12.411 INFO:teuthology.orchestra.run.smithi159.stdout:4327 write 0x3929ad7 thru 0x3939743 (0xfc6d bytes) 2024-09-17T22:58:12.413 INFO:teuthology.orchestra.run.smithi159.stdout:4328 punch from 0x1629d2d to 0x1637f00, (0xe1d3 bytes) 2024-09-17T22:58:12.413 INFO:teuthology.orchestra.run.smithi159.stdout:4329 write 0xb69d26b thru 0xb6a4202 (0x6f98 bytes) 2024-09-17T22:58:12.417 INFO:teuthology.orchestra.run.smithi159.stdout:4330 write 0x1312695 thru 0x131a5ee (0x7f5a bytes) 2024-09-17T22:58:12.418 INFO:teuthology.orchestra.run.smithi159.stdout:4333 writesame 0x2481aa5 thru 0x2485d8b data_size 0x21(0x42e7 bytes) 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164299958999997, 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164283953999998, 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164251184000001, 2024-09-17T22:58:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164223446000001, 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164201026999997, 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.164178747000001, 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.163971342000004, 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.163424726999999, 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.162353531000001, 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.162342076000002, 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.162331098999999, 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.160256967000002, 2024-09-17T22:58:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.160244652999999, 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.160231416000002, 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.160220586999998, 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.158261711999998, 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.158237354000001, 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.158208135999999, 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.157441568000003, 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.157428465999999, 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.516 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.518 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:12.518 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:12.515+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4334 writesame 0xa1129f5 thru 0xa112e83 data_size 0x1(0x48f bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4337 writesame 0x25ea616 thru 0x25ebb4f data_size 0xd(0x153a bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4338 write 0xe6058ac thru 0xe60f531 (0x9c86 bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4339 write 0x74c4e98 thru 0x74c5955 (0xabe bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4340 punch from 0x97007df to 0x9703eeb, (0x370c bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4342 read 0x6304025 thru 0x630f4e4 (0xb4c0 bytes) 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4343 trunc from 0xe60f532 to 0xb09730a 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.520 INFO:teuthology.orchestra.run.smithi159.stdout:4345 trunc from 0xb09730a to 0x536e14a 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4346 trunc from 0x536e14a to 0xa18620b 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4347 writesame 0x63fc921 thru 0x6404187 data_size 0x1(0x7867 bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4349 read 0x94c5835 thru 0x94ce47d (0x8c49 bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4350 punch from 0x2a80ad8 to 0x2a8c427, (0xb94f bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4351 read 0x19c25a2 thru 0x19cf034 (0xca93 bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4353 read 0x9744a17 thru 0x97542dd (0xf8c7 bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4354 trunc from 0xa18620b to 0xca039f2 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4357 write 0xc68efac thru 0xc6957aa (0x67ff bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4358 writesame 0x74f7b8f thru 0x74f931f data_size 0x1(0x1791 bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4360 writesame 0xa0901c thru 0xa157e7 data_size 0x26(0xc7cc bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4361 write 0xc8f3fa6 thru 0xc8ff901 (0xb95c bytes) 2024-09-17T22:58:12.521 INFO:teuthology.orchestra.run.smithi159.stdout:4362 write 0x222415a thru 0x22333eb (0xf292 bytes) 2024-09-17T22:58:12.522 INFO:teuthology.orchestra.run.smithi159.stdout:4363 writesame 0x3193a1e thru 0x319a91a data_size 0x29(0x6efd bytes) 2024-09-17T22:58:12.522 INFO:teuthology.orchestra.run.smithi159.stdout:4364 write 0x8bc09fc thru 0x8bd0833 (0xfe38 bytes) 2024-09-17T22:58:12.522 INFO:teuthology.orchestra.run.smithi159.stdout:4365 writesame 0x7674784 thru 0x767a4d0 data_size 0x11(0x5d4d bytes) 2024-09-17T22:58:12.522 INFO:teuthology.orchestra.run.smithi159.stdout:4366 punch from 0xc086fce to 0xc0894f9, (0x252b bytes) 2024-09-17T22:58:12.522 INFO:teuthology.orchestra.run.smithi159.stdout:4367 write 0x6ba1a3b thru 0x6bb149f (0xfa65 bytes) 2024-09-17T22:58:12.534 INFO:teuthology.orchestra.run.smithi159.stdout:4368 trunc from 0xca039f2 to 0x54792c4 2024-09-17T22:58:12.544 INFO:teuthology.orchestra.run.smithi159.stdout:4369 read 0x2f89f4 thru 0x2fb3ad (0x29ba bytes) 2024-09-17T22:58:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:4370 read 0x53ca4a2 thru 0x53d68a8 (0xc407 bytes) 2024-09-17T22:58:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:4371 read 0x338ddb4 thru 0x339a10d (0xc35a bytes) 2024-09-17T22:58:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:4372 read 0x3562787 thru 0x3569cba (0x7534 bytes) 2024-09-17T22:58:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:4374 read 0x613c2f thru 0x614aa4 (0xe76 bytes) 2024-09-17T22:58:12.557 INFO:teuthology.orchestra.run.smithi159.stdout:4376 write 0xa0ffe7f thru 0xa1037f3 (0x3975 bytes) 2024-09-17T22:58:12.571 INFO:teuthology.orchestra.run.smithi159.stdout:4378 writesame 0xd86db89 thru 0xd871642 data_size 0x1(0x3aba bytes) 2024-09-17T22:58:12.571 INFO:teuthology.orchestra.run.smithi159.stdout:4379 writesame 0xb990f09 thru 0xb99d045 data_size 0x7(0xc13d bytes) 2024-09-17T22:58:12.577 INFO:teuthology.orchestra.run.smithi159.stdout:4381 read 0x6f97763 thru 0x6fa54f0 (0xdd8e bytes) 2024-09-17T22:58:12.577 INFO:teuthology.orchestra.run.smithi159.stdout:4382 read 0x98b7de4 thru 0x98c0a4f (0x8c6c bytes) 2024-09-17T22:58:12.577 INFO:teuthology.orchestra.run.smithi159.stdout:4383 punch from 0xc1f66ee to 0xc1f8a3c, (0x234e bytes) 2024-09-17T22:58:12.577 INFO:teuthology.orchestra.run.smithi159.stdout:4386 writesame 0xa47131b thru 0xa47644c data_size 0x13(0x5132 bytes) 2024-09-17T22:58:12.580 INFO:teuthology.orchestra.run.smithi159.stdout:4387 punch from 0x9392ac3 to 0x93a1f71, (0xf4ae bytes) 2024-09-17T22:58:12.580 INFO:teuthology.orchestra.run.smithi159.stdout:4388 read 0x38d088d thru 0x38d6dff (0x6573 bytes) 2024-09-17T22:58:12.580 INFO:teuthology.orchestra.run.smithi159.stdout:4389 write 0x157da81 thru 0x1582a93 (0x5013 bytes) 2024-09-17T22:58:12.583 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.583 INFO:teuthology.orchestra.run.smithi159.stdout:4392 read 0xaa6abb3 thru 0xaa6d1f2 (0x2640 bytes) 2024-09-17T22:58:12.583 INFO:teuthology.orchestra.run.smithi159.stdout:4394 write 0xa4d8ebd thru 0xa4d8f8d (0xd1 bytes) 2024-09-17T22:58:12.584 INFO:teuthology.orchestra.run.smithi159.stdout:4398 write 0xa4f9066 thru 0xa4faf5d (0x1ef8 bytes) 2024-09-17T22:58:12.587 INFO:teuthology.orchestra.run.smithi159.stdout:4400 read 0x8a23f26 thru 0x8a2aa55 (0x6b30 bytes) 2024-09-17T22:58:12.587 INFO:teuthology.orchestra.run.smithi159.stdout:4401 writesame 0x5fc2425 thru 0x5fc3f03 data_size 0x1(0x1adf bytes) 2024-09-17T22:58:12.591 INFO:teuthology.orchestra.run.smithi159.stdout:4403 write 0x906a1e6 thru 0x90734c6 (0x92e1 bytes) 2024-09-17T22:58:12.594 INFO:teuthology.orchestra.run.smithi159.stdout:4404 write 0xd6835ab thru 0xd68b85e (0x82b4 bytes) 2024-09-17T22:58:12.613 INFO:teuthology.orchestra.run.smithi159.stdout:4405 trunc from 0xd871643 to 0x88bccc 2024-09-17T22:58:12.618 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:12.615+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:12.643 INFO:teuthology.orchestra.run.smithi159.stdout:4406 write 0x979f586 thru 0x97ad6f9 (0xe174 bytes) 2024-09-17T22:58:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:4408 read 0x40595ea thru 0x405d9fa (0x4411 bytes) 2024-09-17T22:58:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:4409 read 0x8160864 thru 0x817014c (0xf8e9 bytes) 2024-09-17T22:58:12.656 INFO:teuthology.orchestra.run.smithi159.stdout:4411 writesame 0xd3df1aa thru 0xd3ea732 data_size 0x7(0xb589 bytes) 2024-09-17T22:58:12.656 INFO:teuthology.orchestra.run.smithi159.stdout:4413 read 0xbde45d8 thru 0xbde47aa (0x1d3 bytes) 2024-09-17T22:58:12.657 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.663 INFO:teuthology.orchestra.run.smithi159.stdout:4416 trunc from 0xd3ea733 to 0x79def9e 2024-09-17T22:58:12.668 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:12.668 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:12.668 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.672 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:12.672 INFO:teuthology.orchestra.run.smithi159.stdout:4418 write 0x5538900 thru 0x5542fbc (0xa6bd bytes) 2024-09-17T22:58:12.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:12.685 INFO:teuthology.orchestra.run.smithi159.stdout:4419 write 0xb6bc630 thru 0xb6c208d (0x5a5e bytes) 2024-09-17T22:58:12.688 INFO:teuthology.orchestra.run.smithi159.stdout:4421 writesame 0x357d0aa thru 0x358c53e data_size 0x1b(0xf495 bytes) 2024-09-17T22:58:12.696 INFO:teuthology.orchestra.run.smithi159.stdout:4422 trunc from 0xb6c208e to 0x7d9ee32 2024-09-17T22:58:12.718 INFO:teuthology.orchestra.run.smithi159.stdout:4424 write 0xdb6d907 thru 0xdb704ea (0x2be4 bytes) 2024-09-17T22:58:12.719 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:12.716+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:12.721 INFO:teuthology.orchestra.run.smithi159.stdout:4427 read 0x315b5f0 thru 0x316986e (0xe27f bytes) 2024-09-17T22:58:12.721 INFO:teuthology.orchestra.run.smithi159.stdout:4428 write 0x4cea41d thru 0x4cf1fc8 (0x7bac bytes) 2024-09-17T22:58:12.724 INFO:teuthology.orchestra.run.smithi159.stdout:4430 read 0x131f1f1 thru 0x132603d (0x6e4d bytes) 2024-09-17T22:58:12.725 INFO:teuthology.orchestra.run.smithi159.stdout:4431 punch from 0xd738ba7 to 0xd746cf4, (0xe14d bytes) 2024-09-17T22:58:12.725 INFO:teuthology.orchestra.run.smithi159.stdout:4432 write 0x91908a1 thru 0x919acd4 (0xa434 bytes) 2024-09-17T22:58:12.727 INFO:teuthology.orchestra.run.smithi159.stdout:4433 writesame 0x9215f93 thru 0x9220b11 data_size 0x1(0xab7f bytes) 2024-09-17T22:58:12.750 INFO:teuthology.orchestra.run.smithi159.stdout:4435 clone 33 order 18 su 1024 sc 12 2024-09-17T22:58:12.782 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.782 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:12.782 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:12.782 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:12.782 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:12.820 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:12.818+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.895 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.031563684999995, 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:12.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:12.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:12.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:12.902 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804880345000001, 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:58:12.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804845809, 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:58:12.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:58:12.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804791426000001, 2024-09-17T22:58:12.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:12.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:58:12.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804766422, 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.917 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:12.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:58:12.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804745840999999, 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:12.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.927 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804724557999997, 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.928 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:12.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.932 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804706031999999, 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:12.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804687860000001, 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:12.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804670031999997, 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:12.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.804354899000003, 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.803632628000003, 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.803588032, 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:58:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.803560709999999, 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.801741911999997, 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:58:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.801399107000002, 2024-09-17T22:58:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:58:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:58:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.799127974999998, 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.799095881, 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:58:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:58:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.799069342999999, 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:58:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.799050450000003, 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:58:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:12.992 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:12.994 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:58:12.994 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:12.991+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:12.995 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:13.016 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone32 from 0xdb704eb (overlap 0x72b540) to 0x0 2024-09-17T22:58:13.094 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:13.092+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:13.123 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T22:58:13.131 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:13.131 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:13.131 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:13.131 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:13.144 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:13.228 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:13.226+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:13.293 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:13.293 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:13.293 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:13.293 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:13.293 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:13.296 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:13.294+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:13.302 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:13.305 INFO:teuthology.orchestra.run.smithi159.stdout:4436 write 0x9828d7a thru 0x982bd1a (0x2fa1 bytes) 2024-09-17T22:58:13.398 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:13.395+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:13.447 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16075 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.182057698999998, 2024-09-17T22:58:13.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309326999999997, 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16075 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265272+0000", 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265273+0000", 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265275+0000", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.733e-06 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265281+0000", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265285+0000", 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6009999999999997e-06 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318573+0000", 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053288281999999999 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318582+0000", 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3639999999999999e-06 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16079 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181953663000002, 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053313915000000003, 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16079 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.456 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265376+0000", 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265377+0000", 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265379+0000", 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265387+0000", 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8630000000000008e-06 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265392+0000", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.938e-06 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318682+0000", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290114999999999 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318690+0000", 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5199999999999997e-06 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.459 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16080 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181932402000001, 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053306547000000003, 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.460 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16080 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:13.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265398+0000", 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265400+0000", 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.03e-06 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265407+0000", 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4450000000000001e-06 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265411+0000", 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.284e-06 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318700+0000", 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289395000000003 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318704+0000", 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9210000000000002e-06 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16081 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181911059000001, 2024-09-17T22:58:13.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053308150999999998, 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.465 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16081 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265419+0000", 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265420+0000", 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265422+0000", 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265428+0000", 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.64e-06 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265433+0000", 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318719+0000", 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286089000000002 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318727+0000", 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2470000000000008e-06 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16084 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.469 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181841601999999, 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053324415, 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16084 2024-09-17T22:58:13.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265489+0000", 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265491+0000", 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9e-06 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265497+0000", 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1090000000000003e-06 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265502+0000", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6580000000000001e-06 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318804+0000", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053302335999999999 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318813+0000", 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9420000000000001e-06 2024-09-17T22:58:13.473 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181822623999999, 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.474 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:13.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.478 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16086 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181793695000003, 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329058999999998, 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16086 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265536+0000", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265537+0000", 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:58:13.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265539+0000", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265546+0000", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2649999999999999e-06 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265556+0000", 2024-09-17T22:58:13.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.6520000000000001e-06 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318862+0000", 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053305492000000003 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318866+0000", 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0029999999999996e-06 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16087 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181767757000003, 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053329617000000003, 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.483 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16087 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265562+0000", 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265563+0000", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265569+0000", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265578+0000", 2024-09-17T22:58:13.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1530000000000003e-06 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265584+0000", 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318884+0000", 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053300027 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318892+0000", 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4370000000000007e-06 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16088 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181739763000003, 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314245000000003, 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16088 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265590+0000", 2024-09-17T22:58:13.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265591+0000", 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4499999999999997e-07 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265593+0000", 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1919999999999999e-06 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265601+0000", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2770000000000005e-06 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265606+0000", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318901+0000", 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053294980999999998 2024-09-17T22:58:13.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318905+0000", 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6480000000000001e-06 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16089 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181718838999998, 2024-09-17T22:58:13.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319560000000002, 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16089 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265611+0000", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265612+0000", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265618+0000", 2024-09-17T22:58:13.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265626+0000", 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1990000000000006e-06 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265631+0000", 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318923+0000", 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053291941000000002 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318931+0000", 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3280000000000006e-06 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16091 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181672098, 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053321517999999998, 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16091 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265658+0000", 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265660+0000", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9530000000000002e-06 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265669+0000", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265674+0000", 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:58:13.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318971+0000", 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053297341999999998 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318980+0000", 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1419999999999993e-06 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16092 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.18164969, 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314175999999998, 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16092 2024-09-17T22:58:13.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265680+0000", 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265681+0000", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4200000000000001e-07 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265683+0000", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265692+0000", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265697+0000", 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318990+0000", 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053292462999999998 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318995+0000", 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16179 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181626346000002, 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319434999999998, 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16179 2024-09-17T22:58:13.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265704+0000", 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.75e-07 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265706+0000", 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265713+0000", 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1299999999999998e-06 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265720+0000", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5499999999999997e-06 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319010+0000", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053289716000000001 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319023+0000", 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.346e-05 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.508 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16181 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181603656999997, 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310772999999999, 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16181 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.509 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265726+0000", 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265727+0000", 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265729+0000", 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265738+0000", 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8179999999999993e-06 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265743+0000", 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.994e-06 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319033+0000", 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053290525999999998 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319037+0000", 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9260000000000002e-06 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16183 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181581457999997, 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053314935000000001, 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.513 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16183 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265749+0000", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265751+0000", 2024-09-17T22:58:13.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265759+0000", 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0180000000000001e-06 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265771+0000", 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1454000000000001e-05 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319056+0000", 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053285523000000001 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319064+0000", 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5510000000000002e-06 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16185 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:13.517 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181552672000002, 2024-09-17T22:58:13.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053319995000000002, 2024-09-17T22:58:13.518 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.518 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.518 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.520 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16185 2024-09-17T22:58:13.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265777+0000", 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265778+0000", 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265780+0000", 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265789+0000", 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6300000000000004e-06 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265794+0000", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319090+0000", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053296071 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319097+0000", 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5440000000000001e-06 2024-09-17T22:58:13.523 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16187 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181530039000002, 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053331019, 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16187 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265800+0000", 2024-09-17T22:58:13.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265801+0000", 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265803+0000", 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.119e-06 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265809+0000", 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2469999999999997e-06 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265814+0000", 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319123+0000", 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053309306000000001 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319131+0000", 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0940000000000008e-06 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.528 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16189 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181510785999997, 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053340783000000003, 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16189 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265819+0000", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265820+0000", 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:58:13.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265829+0000", 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8119999999999997e-06 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265835+0000", 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265840+0000", 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319155+0000", 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053315077000000002 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319160+0000", 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.181483610000001, 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:13.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.003844614999998, 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:13.537 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:13.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:13.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:13.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:13.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:13.544 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:13.544 INFO:teuthology.orchestra.run.smithi159.stdout:4437 trunc from 0xdb704eb to 0x3407f20 2024-09-17T22:58:13.545 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:13.546 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:13.542+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:13.622 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:13.641 INFO:teuthology.orchestra.run.smithi159.stdout:4438 read 0x17bad03 thru 0x17bf751 (0x4a4f bytes) 2024-09-17T22:58:13.645 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:13.643+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:13.651 INFO:teuthology.orchestra.run.smithi159.stdout:4440 write 0x78b82f8 thru 0x78c3ddc (0xbae5 bytes) 2024-09-17T22:58:13.657 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:13.676 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:13.677 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:13.680 INFO:teuthology.orchestra.run.smithi159.stdout:4441 writesame 0xde8605e thru 0xde896a0 data_size 0x1(0x3643 bytes) 2024-09-17T22:58:13.692 INFO:teuthology.orchestra.run.smithi159.stdout:4442 trunc from 0xde896a1 to 0x57f0867 2024-09-17T22:58:13.693 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:13.694 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:13.704 INFO:teuthology.orchestra.run.smithi159.stdout:4443 writesame 0x31d945d thru 0x31de576 data_size 0x7(0x511a bytes) 2024-09-17T22:58:13.719 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:13.719 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:13.745 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:13.745 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:13.746 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:13.743+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:13.749 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:13.749 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T22:58:13.759 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:13.776 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:13.805 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:13.805 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:13.848 INFO:teuthology.orchestra.run.smithi159.stdout:4446 write 0xc81ccc5 thru 0xc825fd3 (0x930f bytes) 2024-09-17T22:58:13.848 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:13.845+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:13.866 INFO:teuthology.orchestra.run.smithi159.stdout:4448 trunc from 0xc825fd4 to 0xe4bc809 2024-09-17T22:58:13.871 INFO:teuthology.orchestra.run.smithi159.stdout:4450 write 0x7789a8c thru 0x7798a06 (0xef7b bytes) 2024-09-17T22:58:13.899 INFO:teuthology.orchestra.run.smithi159.stdout:4451 trunc from 0xe4bc809 to 0x4fe4f32 2024-09-17T22:58:13.903 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:13.903 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:13.914 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:13.914 INFO:teuthology.orchestra.run.smithi159.stdout:4453 read 0x684d34 thru 0x68ea1e (0x9ceb bytes) 2024-09-17T22:58:13.919 INFO:teuthology.orchestra.run.smithi159.stdout:4454 read 0xe24bf8 thru 0xe2b6c6 (0x6acf bytes) 2024-09-17T22:58:13.923 INFO:teuthology.orchestra.run.smithi159.stdout:4455 read 0x14f7fd1 thru 0x14fc6d5 (0x4705 bytes) 2024-09-17T22:58:13.923 INFO:teuthology.orchestra.run.smithi159.stdout:4456 punch from 0x713861 to 0x71f484, (0xbc23 bytes) 2024-09-17T22:58:13.929 INFO:teuthology.orchestra.run.smithi159.stdout:4457 trunc from 0x4fe4f32 to 0x16cde43 2024-09-17T22:58:13.948 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:13.945+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:13.956 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:13.957 INFO:teuthology.orchestra.run.smithi159.stdout:4460 punch from 0x144c86c to 0x1457c67, (0xb3fb bytes) 2024-09-17T22:58:13.968 INFO:teuthology.orchestra.run.smithi159.stdout:4461 write 0x6bbba18 thru 0x6bc2cda (0x72c3 bytes) 2024-09-17T22:58:13.990 INFO:teuthology.orchestra.run.smithi159.stdout:4462 write 0xa74fd5f thru 0xa75dd2e (0xdfd0 bytes) 2024-09-17T22:58:14.006 INFO:teuthology.orchestra.run.smithi159.stdout:4463 writesame 0x123e933 thru 0x124d80e data_size 0x2(0xeedc bytes) 2024-09-17T22:58:14.048 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:14.045+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:14.059 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.059 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:14.154 INFO:teuthology.orchestra.run.smithi159.stdout:4464 write 0x60c3721 thru 0x60c609e (0x297e bytes) 2024-09-17T22:58:14.154 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:14.152+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:14.175 INFO:teuthology.orchestra.run.smithi159.stdout:4465 trunc from 0xa75dd2f to 0xe836adf 2024-09-17T22:58:14.180 INFO:teuthology.orchestra.run.smithi159.stdout:4466 writesame 0x10fd803 thru 0x1100d52 data_size 0x8(0x3550 bytes) 2024-09-17T22:58:14.250 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:14.247+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:14.264 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.264 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.277 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:14.361 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:14.359+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:14.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:14.423 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.423 INFO:teuthology.orchestra.run.smithi159.stdout:4468 write 0xb58c220 thru 0xb58c9e3 (0x7c4 bytes) 2024-09-17T22:58:14.427 INFO:teuthology.orchestra.run.smithi159.stdout:4469 writesame 0x14eade7 thru 0x14f8324 data_size 0x67(0xd53e bytes) 2024-09-17T22:58:14.431 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.432 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.432 INFO:teuthology.orchestra.run.smithi159.stdout:4473 write 0xc8a519d thru 0xc8b02a4 (0xb108 bytes) 2024-09-17T22:58:14.445 INFO:teuthology.orchestra.run.smithi159.stdout:4475 read 0x40161bc thru 0x401ffbe (0x9e03 bytes) 2024-09-17T22:58:14.445 INFO:teuthology.orchestra.run.smithi159.stdout:4476 read 0x7d12da1 thru 0x7d219cd (0xec2d bytes) 2024-09-17T22:58:14.445 INFO:teuthology.orchestra.run.smithi159.stdout:4477 read 0xae5af64 thru 0xae645fb (0x9698 bytes) 2024-09-17T22:58:14.445 INFO:teuthology.orchestra.run.smithi159.stdout:4478 read 0xbf5751e thru 0xbf672f1 (0xfdd4 bytes) 2024-09-17T22:58:14.446 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.446 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.446 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.446 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.452 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:14.450+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:14.458 INFO:teuthology.orchestra.run.smithi159.stdout:4482 trunc from 0xe836adf to 0x38acfe1 2024-09-17T22:58:14.462 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:14.493 INFO:teuthology.orchestra.run.smithi159.stdout:4483 write 0xb92d26f thru 0xb930a41 (0x37d3 bytes) 2024-09-17T22:58:14.513 INFO:teuthology.orchestra.run.smithi159.stdout:4484 trunc from 0xb930a42 to 0xe294ee9 2024-09-17T22:58:14.517 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.517 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.518 INFO:teuthology.orchestra.run.smithi159.stdout:4485 write 0x6d91575 thru 0x6d9a4f3 (0x8f7f bytes) 2024-09-17T22:58:14.529 INFO:teuthology.orchestra.run.smithi159.stdout:4487 read 0xcaa0869 thru 0xcaa17d7 (0xf6f bytes) 2024-09-17T22:58:14.529 INFO:teuthology.orchestra.run.smithi159.stdout:4488 read 0xebba5a thru 0xec449d (0x8a44 bytes) 2024-09-17T22:58:14.532 INFO:teuthology.orchestra.run.smithi159.stdout:4489 read 0x977877 thru 0x97cda0 (0x552a bytes) 2024-09-17T22:58:14.534 INFO:teuthology.orchestra.run.smithi159.stdout:4490 read 0xc1d6448 thru 0xc1e4276 (0xde2f bytes) 2024-09-17T22:58:14.535 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.535 INFO:teuthology.orchestra.run.smithi159.stdout:4492 punch from 0x89e8101 to 0x89edb4d, (0x5a4c bytes) 2024-09-17T22:58:14.535 INFO:teuthology.orchestra.run.smithi159.stdout:4493 write 0x733fd95 thru 0x73495bf (0x982b bytes) 2024-09-17T22:58:14.547 INFO:teuthology.orchestra.run.smithi159.stdout:4494 writesame 0xbfecf2d thru 0xbffadce data_size 0x2e(0xdea2 bytes) 2024-09-17T22:58:14.553 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:14.551+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:14.561 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.561 INFO:teuthology.orchestra.run.smithi159.stdout:4496 write 0x8ed2cda thru 0x8ede4ae (0xb7d5 bytes) 2024-09-17T22:58:14.577 INFO:teuthology.orchestra.run.smithi159.stdout:4497 read 0x62946c3 thru 0x62a115e (0xca9c bytes) 2024-09-17T22:58:14.577 INFO:teuthology.orchestra.run.smithi159.stdout:4498 write 0x572b864 thru 0x5735d65 (0xa502 bytes) 2024-09-17T22:58:14.599 INFO:teuthology.orchestra.run.smithi159.stdout:4499 trunc from 0xe294ee9 to 0x8e556f8 2024-09-17T22:58:14.616 INFO:teuthology.orchestra.run.smithi159.stdout:4500 read 0x40ec1ff thru 0x40f682a (0xa62c bytes) 2024-09-17T22:58:14.617 INFO:teuthology.orchestra.run.smithi159.stdout:4501 write 0x279fdc5 thru 0x27a7aef (0x7d2b bytes) 2024-09-17T22:58:14.623 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.623 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.630 INFO:teuthology.orchestra.run.smithi159.stdout:4504 write 0x60e7fa thru 0x617c8d (0x9494 bytes) 2024-09-17T22:58:14.637 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:14.655 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:14.652+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:14.756 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:14.753+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:14.838 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.838 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:14.849 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:14.856 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:14.854+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:14.868 INFO:teuthology.orchestra.run.smithi159.stdout:4506 read 0x443ead7 thru 0x444db52 (0xf07c bytes) 2024-09-17T22:58:14.868 INFO:teuthology.orchestra.run.smithi159.stdout:4507 write 0x65a0485 thru 0x65a08a2 (0x41e bytes) 2024-09-17T22:58:14.874 INFO:teuthology.orchestra.run.smithi159.stdout:4508 read 0x40eb4b7 thru 0x40f2e5d (0x79a7 bytes) 2024-09-17T22:58:14.874 INFO:teuthology.orchestra.run.smithi159.stdout:4509 read 0x347c905 thru 0x3485d1d (0x9419 bytes) 2024-09-17T22:58:14.875 INFO:teuthology.orchestra.run.smithi159.stdout:4511 read 0x99df20 thru 0x9a3a26 (0x5b07 bytes) 2024-09-17T22:58:14.877 INFO:teuthology.orchestra.run.smithi159.stdout:4512 punch from 0x650d4a6 to 0x6515f65, (0x8abf bytes) 2024-09-17T22:58:14.877 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.877 INFO:teuthology.orchestra.run.smithi159.stdout:4515 read 0x1f29b95 thru 0x1f2c4b1 (0x291d bytes) 2024-09-17T22:58:14.877 INFO:teuthology.orchestra.run.smithi159.stdout:4516 punch from 0x2bc1ee2 to 0x2bcb7cb, (0x98e9 bytes) 2024-09-17T22:58:14.877 INFO:teuthology.orchestra.run.smithi159.stdout:4517 writesame 0x4ff5d58 thru 0x4fffa33 data_size 0x2(0x9cdc bytes) 2024-09-17T22:58:14.893 INFO:teuthology.orchestra.run.smithi159.stdout:4518 trunc from 0x8e556f8 to 0x820e8bf 2024-09-17T22:58:14.906 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.906 INFO:teuthology.orchestra.run.smithi159.stdout:4521 read 0x5eebeeb thru 0x5ef1d8b (0x5ea1 bytes) 2024-09-17T22:58:14.906 INFO:teuthology.orchestra.run.smithi159.stdout:4523 read 0x35ffdff thru 0x3607d78 (0x7f7a bytes) 2024-09-17T22:58:14.906 INFO:teuthology.orchestra.run.smithi159.stdout:4527 read 0x62d418c thru 0x62de1ec (0xa061 bytes) 2024-09-17T22:58:14.907 INFO:teuthology.orchestra.run.smithi159.stdout:4530 write 0x147b63b thru 0x14855cf (0x9f95 bytes) 2024-09-17T22:58:14.917 INFO:teuthology.orchestra.run.smithi159.stdout:4531 write 0x8db17d6 thru 0x8dbd8a3 (0xc0ce bytes) 2024-09-17T22:58:14.935 INFO:teuthology.orchestra.run.smithi159.stdout:4535 write 0x643c16e thru 0x643d6c1 (0x1554 bytes) 2024-09-17T22:58:14.953 INFO:teuthology.orchestra.run.smithi159.stdout:4537 trunc from 0x8dbd8a4 to 0x575d753 2024-09-17T22:58:14.957 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:14.955+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:14.973 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:14.981 INFO:teuthology.orchestra.run.smithi159.stdout:4540 write 0x7fc84fc thru 0x7fd5d57 (0xd85c bytes) 2024-09-17T22:58:14.995 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:58:14.996 INFO:teuthology.orchestra.run.smithi159.stdout:4541 write 0x40c5b54 thru 0x40c7947 (0x1df4 bytes) 2024-09-17T22:58:14.998 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:14.998 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.009 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:15.014 INFO:teuthology.orchestra.run.smithi159.stdout:4542 trunc from 0x7fd5d58 to 0x20aaa5f 2024-09-17T22:58:15.058 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:15.055+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:15.126 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.127 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:15.180 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:15.178+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:15.241 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.241 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.242 INFO:teuthology.orchestra.run.smithi159.stdout:4544 trunc from 0x20aaa5f to 0x7a72dca 2024-09-17T22:58:15.250 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:15.255 INFO:teuthology.orchestra.run.smithi159.stdout:4545 write 0x9c1a321 thru 0x9c27263 (0xcf43 bytes) 2024-09-17T22:58:15.270 INFO:teuthology.orchestra.run.smithi159.stdout:4548 writesame 0x98d15f8 thru 0x98d4e9a data_size 0x1b(0x38a3 bytes) 2024-09-17T22:58:15.284 INFO:teuthology.orchestra.run.smithi159.stdout:4549 read 0x8d56406 thru 0x8d5a384 (0x3f7f bytes) 2024-09-17T22:58:15.285 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:15.285 INFO:teuthology.orchestra.run.smithi159.stdout:4551 read 0x1c6a22b thru 0x1c71d67 (0x7b3d bytes) 2024-09-17T22:58:15.291 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:15.289+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:15.292 INFO:teuthology.orchestra.run.smithi159.stdout:4552 writesame 0xaeb9342 thru 0xaec5131 data_size 0x10(0xbdf0 bytes) 2024-09-17T22:58:15.301 INFO:teuthology.orchestra.run.smithi159.stdout:4553 write 0xda7428e thru 0xda7a457 (0x61ca bytes) 2024-09-17T22:58:15.319 INFO:teuthology.orchestra.run.smithi159.stdout:4554 writesame 0x87c2c72 thru 0x87d03aa data_size 0x11(0xd739 bytes) 2024-09-17T22:58:15.350 INFO:teuthology.orchestra.run.smithi159.stdout:4555 trunc from 0xda7a458 to 0x1a9b47d 2024-09-17T22:58:15.365 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:15.363+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:15.379 INFO:teuthology.orchestra.run.smithi159.stdout:4557 write 0x43ab506 thru 0x43ba1a9 (0xeca4 bytes) 2024-09-17T22:58:15.392 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:15.392 INFO:teuthology.orchestra.run.smithi159.stdout:4559 read 0x1753dcb thru 0x1762089 (0xe2bf bytes) 2024-09-17T22:58:15.392 INFO:teuthology.orchestra.run.smithi159.stdout:4561 read 0x38be9be thru 0x38c2e6a (0x44ad bytes) 2024-09-17T22:58:15.411 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.411 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.411 INFO:teuthology.orchestra.run.smithi159.stdout:4562 writesame 0xe227b38 thru 0xe2287b0 data_size 0x1(0xc79 bytes) 2024-09-17T22:58:15.412 INFO:teuthology.orchestra.run.smithi159.stdout:4565 write 0xb49858a thru 0xb49f914 (0x738b bytes) 2024-09-17T22:58:15.423 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:15.424 INFO:teuthology.orchestra.run.smithi159.stdout:4566 write 0xe6e097 thru 0xe752b5 (0x721f bytes) 2024-09-17T22:58:15.466 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:15.464+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:15.552 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.552 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.561 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:15.605 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:15.603+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:15.653 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:15.658 INFO:teuthology.orchestra.run.smithi159.stdout:4570 trunc from 0xe2287b1 to 0x9618c54 2024-09-17T22:58:15.670 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.670 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.670 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:15.668+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:15.671 INFO:teuthology.orchestra.run.smithi159.stdout:4571 punch from 0x3f63985 to 0x3f6dcf8, (0xa373 bytes) 2024-09-17T22:58:15.671 INFO:teuthology.orchestra.run.smithi159.stdout:4572 write 0x1d031ca thru 0x1d0802c (0x4e63 bytes) 2024-09-17T22:58:15.682 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:15.688 INFO:teuthology.orchestra.run.smithi159.stdout:4574 clone 34 order 21 su 16384 sc 2 2024-09-17T22:58:15.773 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:15.771+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:15.820 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:15.826 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.826 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.835 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:15.876 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:15.874+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:15.936 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:58:15.944 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:15.944 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:15.963 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:15.978 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:15.975+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:15.980 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:15.980 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:15.984 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:15.984 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:15.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:15.997 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:15.997 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:16.015 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:16.015 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:16.029 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone33 from 0x9618c54 (overlap 0x16cde43) to 0x0 2024-09-17T22:58:16.034 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:16.034 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:16.063 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:16.079 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:16.076+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:16.103 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:16.103 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:16.136 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78970 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103069+0000", 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025400157999999999, 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025417806000000001, 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.137 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78970 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103069+0000", 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103069+0000", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103070+0000", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103072+0000", 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.26e-06 2024-09-17T22:58:16.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103077+0000", 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4639999999999997e-06 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103081+0000", 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7009999999999998e-06 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79002 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104531+0000", 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023938615, 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023991992, 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79002 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104531+0000", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104531+0000", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104531+0000", 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.48e-07 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104532+0000", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2100000000000001e-06 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104535+0000", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2210000000000002e-06 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104540+0000", 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8720000000000001e-06 2024-09-17T22:58:16.143 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79231 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118011+0000", 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010458614999999999, 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010540282999999999, 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.144 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79231 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118011+0000", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118011+0000", 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118011+0000", 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3099999999999997e-07 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118015+0000", 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.095e-06 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118022+0000", 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6329999999999996e-06 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118029+0000", 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5459999999999997e-06 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79296 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122377+0000", 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0060922830000000004, 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062022500000000003, 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79296 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122377+0000", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122377+0000", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122378+0000", 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4199999999999995e-07 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122381+0000", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.331e-06 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122388+0000", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3610000000000003e-06 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122395+0000", 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.477e-06 2024-09-17T22:58:16.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79003 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104726+0000", 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023742948, 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.023881685, 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79003 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104726+0000", 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104726+0000", 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104727+0000", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6499999999999998e-07 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104730+0000", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1889999999999999e-06 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104736+0000", 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8850000000000004e-06 2024-09-17T22:58:16.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104741+0000", 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79233 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118082+0000", 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010387703999999999, 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010554977, 2024-09-17T22:58:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79233 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118082+0000", 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118082+0000", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118082+0000", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2200000000000003e-07 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118086+0000", 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.151e-06 2024-09-17T22:58:16.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118093+0000", 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7540000000000006e-06 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118099+0000", 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8850000000000004e-06 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79298 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122411+0000", 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.006057921, 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062522460000000004, 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79298 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122411+0000", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122411+0000", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122412+0000", 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4199999999999996e-07 2024-09-17T22:58:16.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122414+0000", 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.435e-06 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122421+0000", 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6540000000000002e-06 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122429+0000", 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0660000000000004e-06 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78972 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103162+0000", 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025306953, 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025529099999999999, 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.161 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78972 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103162+0000", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103162+0000", 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103163+0000", 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103165+0000", 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103170+0000", 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4499999999999997e-06 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103173+0000", 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9909999999999998e-06 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79005 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.164 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104755+0000", 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023714773000000001, 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.02397328, 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79005 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104755+0000", 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104755+0000", 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104755+0000", 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104757+0000", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104761+0000", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104766+0000", 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1570000000000002e-06 2024-09-17T22:58:16.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79235 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118112+0000", 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.01035694, 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010645936, 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79235 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118112+0000", 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118112+0000", 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118113+0000", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118115+0000", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3599999999999999e-06 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118121+0000", 2024-09-17T22:58:16.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7239999999999997e-06 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118125+0000", 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3329999999999998e-06 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79299 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122438+0000", 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0060317019999999999, 2024-09-17T22:58:16.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063479640000000002, 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79299 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122438+0000", 2024-09-17T22:58:16.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122438+0000", 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122438+0000", 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122440+0000", 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.091e-06 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122450+0000", 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0304e-05 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122464+0000", 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3495e-05 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78973 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103178+0000", 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025291813999999999, 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025636107000000002, 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78973 2024-09-17T22:58:16.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103178+0000", 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103178+0000", 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103178+0000", 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103179+0000", 2024-09-17T22:58:16.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3799999999999999e-06 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103183+0000", 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7840000000000001e-06 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103186+0000", 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999999e-06 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79006 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104770+0000", 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023698961000000001, 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024071682, 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79006 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.179 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104770+0000", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104770+0000", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104771+0000", 2024-09-17T22:58:16.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104772+0000", 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3489999999999999e-06 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104775+0000", 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2710000000000001e-06 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104778+0000", 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7439999999999999e-06 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79237 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118241+0000", 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010228684, 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010629725, 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79237 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118241+0000", 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118241+0000", 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118241+0000", 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0100000000000005e-07 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118243+0000", 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9180000000000001e-06 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118249+0000", 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.716e-06 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118252+0000", 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5499999999999999e-06 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79301 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122852+0000", 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0056170910000000003, 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006044939, 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79301 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122852+0000", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122852+0000", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122853+0000", 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8899999999999999e-07 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122857+0000", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6650000000000001e-06 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122865+0000", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1999999999999994e-06 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122870+0000", 2024-09-17T22:58:16.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.553e-06 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79008 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104811+0000", 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023658711999999998, 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024114013, 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.189 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79008 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104811+0000", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104811+0000", 2024-09-17T22:58:16.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104811+0000", 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3600000000000003e-07 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104814+0000", 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3740000000000001e-06 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104818+0000", 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8400000000000002e-06 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104822+0000", 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2830000000000002e-06 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79239 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118329+0000", 2024-09-17T22:58:16.192 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.01014028, 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010623416, 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79239 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118329+0000", 2024-09-17T22:58:16.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118329+0000", 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118330+0000", 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118332+0000", 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4530000000000001e-06 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118339+0000", 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.759e-06 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118342+0000", 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1480000000000002e-06 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79303 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122901+0000", 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0055679919999999999, 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006081581, 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.196 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79303 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122901+0000", 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122901+0000", 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122902+0000", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122904+0000", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122909+0000", 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2290000000000002e-06 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122912+0000", 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2820000000000001e-06 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78975 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103292+0000", 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025177304000000001, 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025718866, 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78975 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103292+0000", 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103292+0000", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103293+0000", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103295+0000", 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6230000000000001e-06 2024-09-17T22:58:16.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103300+0000", 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8919999999999999e-06 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103305+0000", 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4139999999999996e-06 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79010 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104920+0000", 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023549736000000002, 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024118713, 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79010 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104920+0000", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104920+0000", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104920+0000", 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104922+0000", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104927+0000", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.42e-06 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104930+0000", 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.926e-06 2024-09-17T22:58:16.205 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79241 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118450+0000", 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010019363, 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010615103000000001, 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.206 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79241 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118450+0000", 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118450+0000", 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118451+0000", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8299999999999996e-07 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118454+0000", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.377e-06 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118461+0000", 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.17e-06 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118470+0000", 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1549999999999996e-06 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79305 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122917+0000", 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0055528180000000002, 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061762379999999997, 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.209 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79305 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122917+0000", 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122917+0000", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122917+0000", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122918+0000", 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:58:16.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122922+0000", 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2300000000000002e-06 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122925+0000", 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6919999999999998e-06 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79012 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105080+0000", 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023389459000000001, 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024041303999999999, 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79012 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105080+0000", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105080+0000", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105081+0000", 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1100000000000005e-07 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105084+0000", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5130000000000001e-06 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105093+0000", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4349999999999997e-06 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105098+0000", 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:58:16.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79244 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118570+0000", 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0098991640000000002, 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010577982, 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.216 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79244 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118570+0000", 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118570+0000", 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118571+0000", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118573+0000", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118580+0000", 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:58:16.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118583+0000", 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9090000000000002e-06 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79307 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122992+0000", 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0054775700000000002, 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061836449999999998, 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79307 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122992+0000", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122992+0000", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122992+0000", 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:58:16.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122994+0000", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.153e-06 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123000+0000", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.152e-06 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123003+0000", 2024-09-17T22:58:16.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.23e-06 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78977 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103397+0000", 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025072587, 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025805899, 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.223 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78977 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103397+0000", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103397+0000", 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103397+0000", 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.58e-07 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103399+0000", 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.154e-06 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103406+0000", 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9e-06 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103411+0000", 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.226 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79014 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105148+0000", 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.02332159, 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024082932000000001, 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79014 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105148+0000", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105148+0000", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105148+0000", 2024-09-17T22:58:16.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105150+0000", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1339999999999998e-06 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105156+0000", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.626e-06 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105160+0000", 2024-09-17T22:58:16.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2599999999999999e-06 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79246 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118639+0000", 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0098303160000000004, 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010618871, 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79246 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118639+0000", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118639+0000", 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118640+0000", 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118642+0000", 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3259999999999998e-06 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118647+0000", 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118650+0000", 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1659999999999998e-06 2024-09-17T22:58:16.233 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79309 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.123101+0000", 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0053682080000000002, 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061837050000000003, 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79309 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123101+0000", 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123101+0000", 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123102+0000", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123104+0000", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3180000000000001e-06 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123109+0000", 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:58:16.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123115+0000", 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0929999999999999e-06 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.237 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79015 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105166+0000", 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023303139, 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024145781000000002, 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79015 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105166+0000", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105166+0000", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105166+0000", 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5899999999999998e-07 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105168+0000", 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3400000000000001e-06 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105172+0000", 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.211e-06 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105175+0000", 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.892e-06 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79248 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118821+0000", 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0096480880000000008, 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010518141, 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.241 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79248 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118821+0000", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118821+0000", 2024-09-17T22:58:16.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118822+0000", 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0200000000000001e-07 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118826+0000", 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7610000000000001e-06 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118833+0000", 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7810000000000005e-06 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118845+0000", 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.151e-05 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79311 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.123272+0000", 2024-09-17T22:58:16.244 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051977300000000002, 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0060953919999999998, 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79311 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123272+0000", 2024-09-17T22:58:16.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123272+0000", 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123272+0000", 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:16.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123275+0000", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8380000000000002e-06 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123281+0000", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.609e-06 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123285+0000", 2024-09-17T22:58:16.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1080000000000003e-06 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78979 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103463+0000", 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025006635999999999, 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.025932345999999998, 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.248 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78979 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103463+0000", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103463+0000", 2024-09-17T22:58:16.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103463+0000", 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103465+0000", 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103471+0000", 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3249999999999998e-06 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103476+0000", 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79017 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105280+0000", 2024-09-17T22:58:16.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023189821999999999, 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024142771, 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79017 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105280+0000", 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105280+0000", 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105280+0000", 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5899999999999996e-07 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105282+0000", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2639999999999998e-06 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105287+0000", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7469999999999996e-06 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105291+0000", 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9639999999999997e-06 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.254 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79250 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.118987+0000", 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.009482061, 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010462321, 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.255 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79250 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118987+0000", 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118987+0000", 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118988+0000", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4300000000000003e-07 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118990+0000", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4650000000000001e-06 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.118997+0000", 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3659999999999997e-06 2024-09-17T22:58:16.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119000+0000", 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8160000000000004e-06 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.258 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79313 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.123357+0000", 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051122959999999997, 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061383480000000001, 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79313 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123357+0000", 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123357+0000", 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123358+0000", 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123359+0000", 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.902e-06 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123364+0000", 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9479999999999999e-06 2024-09-17T22:58:16.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123368+0000", 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3749999999999999e-06 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79019 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105362+0000", 2024-09-17T22:58:16.262 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023106874999999999, 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024161037, 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79019 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105362+0000", 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105362+0000", 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105363+0000", 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3200000000000004e-07 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105366+0000", 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8140000000000001e-06 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105373+0000", 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6989999999999997e-06 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105379+0000", 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3759999999999996e-06 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79252 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119182+0000", 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0092868949999999999, 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010368263000000001, 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.266 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79252 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119182+0000", 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119182+0000", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119183+0000", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9500000000000002e-07 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119187+0000", 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3759999999999999e-06 2024-09-17T22:58:16.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119195+0000", 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5429999999999994e-06 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119201+0000", 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.857e-06 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79315 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.123792+0000", 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.004677507, 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0057864850000000001, 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79315 2024-09-17T22:58:16.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123792+0000", 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123792+0000", 2024-09-17T22:58:16.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123792+0000", 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5099999999999999e-07 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123796+0000", 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.05e-06 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123803+0000", 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0620000000000003e-06 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123809+0000", 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78949 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.101860+0000", 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.026608881000000001, 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027745690999999999, 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78949 2024-09-17T22:58:16.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101860+0000", 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101860+0000", 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101861+0000", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4900000000000005e-07 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101865+0000", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6459999999999999e-06 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101873+0000", 2024-09-17T22:58:16.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.101880+0000", 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5950000000000001e-06 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.277 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78981 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103529+0000", 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024940431999999998, 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026105336, 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.278 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78981 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103529+0000", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103529+0000", 2024-09-17T22:58:16.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103529+0000", 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103531+0000", 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103535+0000", 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.687e-06 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103538+0000", 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9610000000000001e-06 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79021 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.105473+0000", 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.022995993999999999, 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.024188169999999998, 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79021 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.282 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105473+0000", 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105473+0000", 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105474+0000", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6099999999999999e-07 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105477+0000", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1860000000000001e-06 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105485+0000", 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:16.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.105491+0000", 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2360000000000001e-06 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79254 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119277+0000", 2024-09-17T22:58:16.285 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0091918920000000001, 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010411137000000001, 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79254 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119277+0000", 2024-09-17T22:58:16.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119277+0000", 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119278+0000", 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8899999999999999e-07 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119281+0000", 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.128e-06 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119288+0000", 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0310000000000001e-06 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119294+0000", 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.57e-06 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79317 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.123817+0000", 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0046521569999999996, 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059043059999999998, 2024-09-17T22:58:16.289 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79317 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123817+0000", 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123817+0000", 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123817+0000", 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.67e-07 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123819+0000", 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.392e-06 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123826+0000", 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7209999999999997e-06 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.123833+0000", 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0890000000000002e-06 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79256 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119402+0000", 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0090676980000000008, 2024-09-17T22:58:16.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010347238999999999, 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79256 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119402+0000", 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119402+0000", 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119402+0000", 2024-09-17T22:58:16.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6199999999999998e-07 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119405+0000", 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7149999999999998e-06 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119410+0000", 2024-09-17T22:58:16.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4859999999999997e-06 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119414+0000", 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.399e-06 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78951 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102413+0000", 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.026056267000000001, 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027363264000000002, 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78951 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102413+0000", 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102413+0000", 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102414+0000", 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102417+0000", 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9469999999999999e-06 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102423+0000", 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5529999999999998e-06 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102428+0000", 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.276e-06 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78983 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103675+0000", 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024794172, 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026128408999999998, 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78983 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.302 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103675+0000", 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103675+0000", 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103676+0000", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5300000000000004e-07 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103678+0000", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4250000000000001e-06 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103683+0000", 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7829999999999998e-06 2024-09-17T22:58:16.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103686+0000", 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.196e-06 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79258 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.305 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119610+0000", 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0088595610000000002, 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010220913999999999, 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79258 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119610+0000", 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119610+0000", 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119610+0000", 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8999999999999996e-07 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119614+0000", 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3390000000000001e-06 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119622+0000", 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119631+0000", 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0119999999999994e-06 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79260 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119687+0000", 2024-09-17T22:58:16.309 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0087828149999999994, 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.01017216, 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79260 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119687+0000", 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119687+0000", 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119687+0000", 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4300000000000003e-07 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119690+0000", 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6220000000000001e-06 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119695+0000", 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119699+0000", 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9149999999999998e-06 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79262 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119864+0000", 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0086053589999999999, 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.010021542, 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79262 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.314 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119864+0000", 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119864+0000", 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119864+0000", 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119866+0000", 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119872+0000", 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9869999999999996e-06 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119876+0000", 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3299999999999999e-06 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.317 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78954 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102492+0000", 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025977100999999999, 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027420636000000002, 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.318 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78954 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102492+0000", 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102492+0000", 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102493+0000", 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102495+0000", 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0370000000000001e-06 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102502+0000", 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102507+0000", 2024-09-17T22:58:16.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1150000000000002e-06 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78986 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103763+0000", 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024705886999999999, 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026176906, 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78986 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103763+0000", 2024-09-17T22:58:16.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103763+0000", 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103764+0000", 2024-09-17T22:58:16.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103766+0000", 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.03e-06 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103772+0000", 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1700000000000002e-06 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103777+0000", 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9030000000000003e-06 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79264 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.119991+0000", 2024-09-17T22:58:16.326 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0084786659999999993, 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0099780700000000003, 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79264 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119991+0000", 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119991+0000", 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119991+0000", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5499999999999998e-07 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.119995+0000", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5329999999999999e-06 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120004+0000", 2024-09-17T22:58:16.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9080000000000001e-06 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120016+0000", 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2605e-05 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79266 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120159+0000", 2024-09-17T22:58:16.330 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00830995, 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0098363519999999996, 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79266 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120159+0000", 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120159+0000", 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120160+0000", 2024-09-17T22:58:16.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120163+0000", 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.841e-06 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120169+0000", 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.285e-06 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120173+0000", 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7459999999999998e-06 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78956 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102533+0000", 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025936333999999998, 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027489902, 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78956 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102533+0000", 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102533+0000", 2024-09-17T22:58:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102533+0000", 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102535+0000", 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0430000000000002e-06 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102541+0000", 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8919999999999997e-06 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102547+0000", 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3279999999999996e-06 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78988 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103839+0000", 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024630098, 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026210805, 2024-09-17T22:58:16.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78988 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103839+0000", 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103839+0000", 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103840+0000", 2024-09-17T22:58:16.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3200000000000005e-07 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103843+0000", 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8839999999999998e-06 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103850+0000", 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.097e-06 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103855+0000", 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.356e-06 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79268 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120300+0000", 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0081688960000000001, 2024-09-17T22:58:16.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0097762679999999994, 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79268 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120300+0000", 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120300+0000", 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120301+0000", 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7899999999999999e-07 2024-09-17T22:58:16.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120303+0000", 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.447e-06 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120308+0000", 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9629999999999997e-06 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120313+0000", 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8709999999999996e-06 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78989 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.103918+0000", 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024551106, 2024-09-17T22:58:16.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026185534999999999, 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78989 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103918+0000", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103918+0000", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103919+0000", 2024-09-17T22:58:16.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7999999999999995e-07 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103922+0000", 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103927+0000", 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0799999999999996e-06 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.103930+0000", 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6179999999999999e-06 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79270 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120502+0000", 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0079677810000000002, 2024-09-17T22:58:16.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096289879999999998, 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79270 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120502+0000", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120502+0000", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120502+0000", 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8100000000000003e-07 2024-09-17T22:58:16.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120505+0000", 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.864e-06 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120511+0000", 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9359999999999996e-06 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120515+0000", 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.036e-06 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78958 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102583+0000", 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025886493999999999, 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027574973999999999, 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78958 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102583+0000", 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102583+0000", 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102583+0000", 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102585+0000", 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0470000000000001e-06 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102591+0000", 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.592e-06 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102597+0000", 2024-09-17T22:58:16.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79272 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120584+0000", 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0078853870000000006, 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0096013969999999994, 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.359 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79272 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120584+0000", 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120584+0000", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120585+0000", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6199999999999998e-07 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120587+0000", 2024-09-17T22:58:16.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.565e-06 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120592+0000", 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6480000000000002e-06 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120595+0000", 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.343e-06 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78991 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104036+0000", 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024433680999999999, 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026176723999999998, 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78991 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104036+0000", 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104036+0000", 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104036+0000", 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104039+0000", 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4269999999999998e-06 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104044+0000", 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9609999999999996e-06 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104047+0000", 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0939999999999999e-06 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79274 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.366 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120794+0000", 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0076754249999999996, 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094453239999999997, 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79274 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120794+0000", 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120794+0000", 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120795+0000", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6199999999999997e-07 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120798+0000", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7910000000000002e-06 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120808+0000", 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.713e-06 2024-09-17T22:58:16.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120815+0000", 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4059999999999999e-06 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78960 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102626+0000", 2024-09-17T22:58:16.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025842898999999999, 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027640827, 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78960 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102626+0000", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102626+0000", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102627+0000", 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:58:16.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102629+0000", 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2649999999999999e-06 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102634+0000", 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102637+0000", 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9610000000000001e-06 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79276 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120943+0000", 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0075268059999999996, 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093581080000000004, 2024-09-17T22:58:16.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79276 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120943+0000", 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120943+0000", 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120943+0000", 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1399999999999997e-07 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120946+0000", 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.903e-06 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120952+0000", 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3149999999999997e-06 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120956+0000", 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.378 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78993 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104125+0000", 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024344434000000002, 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026203476, 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78993 2024-09-17T22:58:16.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104125+0000", 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104125+0000", 2024-09-17T22:58:16.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104125+0000", 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:16.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104127+0000", 2024-09-17T22:58:16.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5430000000000001e-06 2024-09-17T22:58:16.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104130+0000", 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6009999999999999e-06 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104133+0000", 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79217 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117273+0000", 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011196826, 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013083068999999999, 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79217 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117273+0000", 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117273+0000", 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117273+0000", 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7700000000000004e-07 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117276+0000", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6970000000000002e-06 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117288+0000", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2340999999999999e-05 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117294+0000", 2024-09-17T22:58:16.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79278 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.120963+0000", 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0075064279999999999, 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094198040000000004, 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.390 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79278 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120963+0000", 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120963+0000", 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120963+0000", 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120965+0000", 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4219999999999999e-06 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120969+0000", 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.120971+0000", 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.553e-06 2024-09-17T22:58:16.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78962 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102669+0000", 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025800362, 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027740814999999999, 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78962 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102669+0000", 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102669+0000", 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102669+0000", 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0499999999999999e-07 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102671+0000", 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.457e-06 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102674+0000", 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.546e-06 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102677+0000", 2024-09-17T22:58:16.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79218 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117305+0000", 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011164399, 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013215667, 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79218 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117305+0000", 2024-09-17T22:58:16.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117305+0000", 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117306+0000", 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.37e-07 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117308+0000", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.323e-06 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117314+0000", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117319+0000", 2024-09-17T22:58:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9069999999999998e-06 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79280 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121157+0000", 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0073126179999999999, 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093915720000000008, 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79280 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121157+0000", 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121157+0000", 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121157+0000", 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1999999999999999e-07 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121161+0000", 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1820000000000002e-06 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121168+0000", 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4030000000000001e-06 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121176+0000", 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.746e-06 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78995 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104263+0000", 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024206575000000001, 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026325893, 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.406 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78995 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104263+0000", 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104263+0000", 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104263+0000", 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8599999999999998e-07 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104267+0000", 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5549999999999998e-06 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104275+0000", 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0590000000000003e-06 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104286+0000", 2024-09-17T22:58:16.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0832999999999999e-05 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79219 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117421+0000", 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011048675000000001, 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013225043000000001, 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.410 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79219 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117421+0000", 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117421+0000", 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117421+0000", 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6599999999999996e-07 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117424+0000", 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7159999999999999e-06 2024-09-17T22:58:16.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117429+0000", 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117433+0000", 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.591e-06 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.413 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79282 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121297+0000", 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0071727550000000003, 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094068759999999998, 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79282 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121297+0000", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121297+0000", 2024-09-17T22:58:16.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121297+0000", 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121300+0000", 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5620000000000002e-06 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121306+0000", 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7070000000000003e-06 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121310+0000", 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8689999999999997e-06 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78964 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102846+0000", 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025623798, 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.027920505000000002, 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.418 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78964 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102846+0000", 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102846+0000", 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102846+0000", 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4200000000000002e-07 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102847+0000", 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.238e-06 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102850+0000", 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1209999999999998e-06 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102854+0000", 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0409999999999999e-06 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79220 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117442+0000", 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.011027037999999999, 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013380484, 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79220 2024-09-17T22:58:16.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117442+0000", 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117442+0000", 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117443+0000", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117445+0000", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117448+0000", 2024-09-17T22:58:16.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7950000000000001e-06 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117452+0000", 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4130000000000002e-06 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79284 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121369+0000", 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0071002929999999997, 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094956209999999992, 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79284 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121369+0000", 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121369+0000", 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121370+0000", 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2300000000000001e-07 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121372+0000", 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5170000000000002e-06 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121377+0000", 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7720000000000002e-06 2024-09-17T22:58:16.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121380+0000", 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.275e-06 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78997 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104457+0000", 2024-09-17T22:58:16.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.024012233000000001, 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026439087, 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78997 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104457+0000", 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104457+0000", 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104458+0000", 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.75e-07 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104460+0000", 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6299999999999998e-06 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104466+0000", 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4750000000000001e-06 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104469+0000", 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4879999999999999e-06 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79221 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117472+0000", 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010997249000000001, 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013452196, 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.433 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79221 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117472+0000", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117472+0000", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117472+0000", 2024-09-17T22:58:16.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.27e-07 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117474+0000", 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5629999999999999e-06 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117478+0000", 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0550000000000001e-06 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117482+0000", 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2640000000000002e-06 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79286 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121524+0000", 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0069456170000000003, 2024-09-17T22:58:16.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094289939999999996, 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79286 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121524+0000", 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121524+0000", 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121524+0000", 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7800000000000001e-07 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121527+0000", 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9299999999999999e-06 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121534+0000", 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121540+0000", 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78966 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102874+0000", 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025595584000000001, 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028112083, 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78966 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102874+0000", 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102874+0000", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102874+0000", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102876+0000", 2024-09-17T22:58:16.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9259999999999999e-06 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102882+0000", 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4609999999999999e-06 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102887+0000", 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8320000000000002e-06 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79223 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117643+0000", 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010825952, 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013392895, 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.446 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79223 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117643+0000", 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117643+0000", 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117644+0000", 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117647+0000", 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0699999999999998e-06 2024-09-17T22:58:16.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117655+0000", 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2830000000000001e-06 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117662+0000", 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2319999999999998e-06 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79288 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121677+0000", 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0067924209999999999, 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094131509999999998, 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79288 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121677+0000", 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121677+0000", 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121677+0000", 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0200000000000002e-07 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121680+0000", 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6419999999999999e-06 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121687+0000", 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4710000000000004e-06 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121690+0000", 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9439999999999998e-06 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78999 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104498+0000", 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023971176, 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026645417000000001, 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78999 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104498+0000", 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104498+0000", 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104498+0000", 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2600000000000001e-07 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104500+0000", 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2720000000000001e-06 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104503+0000", 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.038e-06 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104505+0000", 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6060000000000001e-06 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79225 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117689+0000", 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010780814, 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013512372999999999, 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79225 2024-09-17T22:58:16.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117689+0000", 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117689+0000", 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117689+0000", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9899999999999999e-07 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117692+0000", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.05e-06 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117699+0000", 2024-09-17T22:58:16.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2349999999999996e-06 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117705+0000", 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.302e-06 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79290 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121749+0000", 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0067200790000000003, 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0094925659999999992, 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79290 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121749+0000", 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121749+0000", 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121750+0000", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6700000000000003e-07 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121752+0000", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121759+0000", 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.669e-06 2024-09-17T22:58:16.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121764+0000", 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6020000000000002e-06 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:78968 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.102922+0000", 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.025547797000000001, 2024-09-17T22:58:16.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.028350856000000001, 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 78968 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102922+0000", 2024-09-17T22:58:16.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102922+0000", 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102922+0000", 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:58:16.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102924+0000", 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5210000000000001e-06 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102931+0000", 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1489999999999999e-06 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.102936+0000", 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79000 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.104509+0000", 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.023960604999999999, 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.026791194000000001, 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79000 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104509+0000", 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104509+0000", 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104509+0000", 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104510+0000", 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4270000000000001e-06 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104513+0000", 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0369999999999999e-06 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.104516+0000", 2024-09-17T22:58:16.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5450000000000002e-06 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79227 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117743+0000", 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010726256, 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013585554, 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.473 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79227 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117743+0000", 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117743+0000", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117743+0000", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117746+0000", 2024-09-17T22:58:16.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4260000000000002e-06 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117751+0000", 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.605e-06 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117754+0000", 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.163e-06 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79292 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.121977+0000", 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0064919039999999997, 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093793820000000003, 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79292 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121977+0000", 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121977+0000", 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121978+0000", 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121982+0000", 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5090000000000002e-06 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121989+0000", 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6070000000000002e-06 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.121995+0000", 2024-09-17T22:58:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0190000000000003e-06 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79229 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.117871+0000", 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.010598629, 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.013534667, 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79229 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117871+0000", 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117871+0000", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117871+0000", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8899999999999999e-07 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117874+0000", 2024-09-17T22:58:16.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4140000000000001e-06 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117882+0000", 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4999999999999999e-06 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.117886+0000", 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8670000000000004e-06 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:79294 3.f 3.bfc7510f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:16.122149+0000", 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.006320015, 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0093097419999999993, 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:16.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 79294 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122149+0000", 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122149+0000", 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122150+0000", 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5899999999999996e-07 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122153+0000", 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.875e-06 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122158+0000", 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2249999999999999e-06 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:16.122162+0000", 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.749e-06 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.488 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T22:58:16.489 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 92 2024-09-17T22:58:16.489 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:16.490 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:16.487+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:16.490 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:16.591 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:16.588+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:16.662 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:16.662 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:16.662 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:16.662 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:16.662 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:16.672 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:16.715 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:16.713+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:16.780 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:16.780 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:16.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:16.780 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:16.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 72.916253252999994, 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.781 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:16.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:16.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:16.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689569913, 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:16.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689535376999999, 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.793 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.796 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:16.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689480994, 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:58:16.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689455989999999, 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.802 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:58:16.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:58:16.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.806 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689435408999998, 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:58:16.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:58:16.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:58:16.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:16.811 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689414126000003, 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:16.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:16.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:16.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689395599999997, 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:16.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689377428, 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:16.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689359600000003, 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:16.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:16.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.689044467000002, 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.830 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:16.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:16.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.688322196000001, 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:58:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:58:16.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.688277599999999, 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:58:16.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.688250277999998, 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.686431480000003, 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:58:16.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:58:16.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.686088675000001, 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:58:16.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:58:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:58:16.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.683817543000004, 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:58:16.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.683785448999998, 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.864 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:58:16.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.683758910999998, 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:58:16.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.683740018000002, 2024-09-17T22:58:16.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:16.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:58:16.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:16.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:16.878 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:16.880 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:16.880 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:16.877+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:16.883 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T22:58:16.884 INFO:teuthology.orchestra.run.smithi159.stdout:4575 writesame 0x7d4d5a7 thru 0x7d5a991 data_size 0x1(0xd3eb bytes) 2024-09-17T22:58:16.920 INFO:teuthology.orchestra.run.smithi159.stdout:4576 write 0xa23390a thru 0xa23cf74 (0x966b bytes) 2024-09-17T22:58:16.924 INFO:teuthology.orchestra.run.smithi159.stdout:4578 read 0x1a23026 thru 0x1a26a20 (0x39fb bytes) 2024-09-17T22:58:16.924 INFO:teuthology.orchestra.run.smithi159.stdout:4582 read 0x955fe2c thru 0x956015f (0x334 bytes) 2024-09-17T22:58:16.924 INFO:teuthology.orchestra.run.smithi159.stdout:4583 write 0x47dd840 thru 0x47e2152 (0x4913 bytes) 2024-09-17T22:58:16.969 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:16.980 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:16.978+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:16.981 INFO:teuthology.orchestra.run.smithi159.stdout:4585 write 0xe89a4e7 thru 0xe89c308 (0x1e22 bytes) 2024-09-17T22:58:16.984 INFO:teuthology.orchestra.run.smithi159.stdout:4586 read 0xc88b06c thru 0xc88e5b0 (0x3545 bytes) 2024-09-17T22:58:16.984 INFO:teuthology.orchestra.run.smithi159.stdout:4587 trunc from 0xe89c309 to 0xe297e82 2024-09-17T22:58:16.994 INFO:teuthology.orchestra.run.smithi159.stdout:4588 punch from 0xb2dd504 to 0xb2e15b8, (0x40b4 bytes) 2024-09-17T22:58:16.994 INFO:teuthology.orchestra.run.smithi159.stdout:4589 read 0xb30efd1 thru 0xb31ae76 (0xbea6 bytes) 2024-09-17T22:58:17.007 INFO:teuthology.orchestra.run.smithi159.stdout:4590 trunc from 0xe297e82 to 0x3323683 2024-09-17T22:58:17.031 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:17.041 INFO:teuthology.orchestra.run.smithi159.stdout:4594 write 0x52428ce thru 0x5250a3e (0xe171 bytes) 2024-09-17T22:58:17.044 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:17.044 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:17.044 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:17.053 INFO:teuthology.orchestra.run.smithi159.stdout:4595 write 0x5634d09 thru 0x563d712 (0x8a0a bytes) 2024-09-17T22:58:17.054 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:17.058 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:17.072 INFO:teuthology.orchestra.run.smithi159.stdout:4598 write 0xb152379 thru 0xb155f22 (0x3baa bytes) 2024-09-17T22:58:17.075 INFO:teuthology.orchestra.run.smithi159.stdout:4599 writesame 0x6ac612e thru 0x6ad2ec1 data_size 0x4(0xcd94 bytes) 2024-09-17T22:58:17.082 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:17.082 INFO:teuthology.orchestra.run.smithi159.stdout:4603 write 0x62c2f56 thru 0x62ca0f1 (0x719c bytes) 2024-09-17T22:58:17.096 INFO:teuthology.orchestra.run.smithi159.stdout:4604 writesame 0xd18ca95 thru 0xd190d20 data_size 0x2(0x428c bytes) 2024-09-17T22:58:17.096 INFO:teuthology.orchestra.run.smithi159.stdout:4605 read 0xa55480b thru 0xa561ee4 (0xd6da bytes) 2024-09-17T22:58:17.096 INFO:teuthology.orchestra.run.smithi159.stdout:4606 writesame 0x13fd373 thru 0x13fffe7 data_size 0x1(0x2c75 bytes) 2024-09-17T22:58:17.097 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:17.095+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:17.183 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:17.180+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:17.183 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:17.183 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:17.183 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:17.184 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:17.184 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:17.192 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:17.192 INFO:teuthology.orchestra.run.smithi159.stdout:4607 trunc from 0xd190d21 to 0x85ab507 2024-09-17T22:58:17.205 INFO:teuthology.orchestra.run.smithi159.stdout:4608 read 0x7cfa3a7 thru 0x7d063bc (0xc016 bytes) 2024-09-17T22:58:17.205 INFO:teuthology.orchestra.run.smithi159.stdout:4609 read 0x20cd382 thru 0x20ce779 (0x13f8 bytes) 2024-09-17T22:58:17.205 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:17.206 INFO:teuthology.orchestra.run.smithi159.stdout:4614 trunc from 0x85ab507 to 0x7c5bba2 2024-09-17T22:58:17.208 INFO:teuthology.orchestra.run.smithi159.stdout:4616 write 0x3fba8fe thru 0x3fc9492 (0xeb95 bytes) 2024-09-17T22:58:17.225 INFO:teuthology.orchestra.run.smithi159.stdout:4617 write 0xcc81550 thru 0xcc9127b (0xfd2c bytes) 2024-09-17T22:58:17.228 INFO:teuthology.orchestra.run.smithi159.stdout:4618 write 0xa554502 thru 0xa5600ef (0xbbee bytes) 2024-09-17T22:58:17.242 INFO:teuthology.orchestra.run.smithi159.stdout:4619 write 0xd920bc1 thru 0xd9296fa (0x8b3a bytes) 2024-09-17T22:58:17.249 INFO:teuthology.orchestra.run.smithi159.stdout:4622 read 0x79e13f7 thru 0x79ee687 (0xd291 bytes) 2024-09-17T22:58:17.249 INFO:teuthology.orchestra.run.smithi159.stdout:4626 punch from 0xa8b2b96 to 0xa8b428b, (0x16f5 bytes) 2024-09-17T22:58:17.249 INFO:teuthology.orchestra.run.smithi159.stdout:4627 read 0xacabfc7 thru 0xacb196f (0x59a9 bytes) 2024-09-17T22:58:17.250 INFO:teuthology.orchestra.run.smithi159.stdout:4630 trunc from 0xd9296fb to 0xeac31e9 2024-09-17T22:58:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:4632 trunc from 0xeac31e9 to 0x23cc8e1 2024-09-17T22:58:17.283 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:17.280+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:4633 punch from 0x14be842 to 0x14c0673, (0x1e31 bytes) 2024-09-17T22:58:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:4634 writesame 0x705cd4b thru 0x706863c data_size 0x27(0xb8f2 bytes) 2024-09-17T22:58:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:4636 trunc from 0x706863d to 0x9edd99e 2024-09-17T22:58:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:4637 trunc from 0x9edd99e to 0x2537783 2024-09-17T22:58:17.344 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:17.344 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:58:17.344 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:4638 writesame 0x709a46f thru 0x70a33e7 data_size 0x35(0x8f79 bytes) 2024-09-17T22:58:17.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:4640 trunc from 0x70a33e8 to 0x450e8f7 2024-09-17T22:58:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:4642 read 0x35a841a thru 0x35b591b (0xd502 bytes) 2024-09-17T22:58:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:4645 punch from 0x2f78c93 to 0x2f8372f, (0xaa9c bytes) 2024-09-17T22:58:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:4647 punch from 0x2fb238c to 0x2fbc6b5, (0xa329 bytes) 2024-09-17T22:58:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:4649 trunc from 0x450e8f7 to 0xea33f12 2024-09-17T22:58:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:4650 write 0xa5420d6 thru 0xa54e4f7 (0xc422 bytes) 2024-09-17T22:58:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:4651 read 0xdb4c9f1 thru 0xdb57dbb (0xb3cb bytes) 2024-09-17T22:58:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:4652 flatten 2024-09-17T22:58:17.383 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:17.381+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:17.484 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:17.482+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:17.532 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:17.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:17.532 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:17.532 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:17.545 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:17.588 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:17.585+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:17.686 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:17.684+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:17.727 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:17.727 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:17.727 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:17.727 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:17.727 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:17.741 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:17.787 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:17.785+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:17.889 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:17.887+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:17.950 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:17.969 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.698029013000003, 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:17.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.973 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.697689998999998, 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.974 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:17.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 33.520051004000003, 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.978 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:17.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:17.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:17.984 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8758607569999999, 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:58:17.988 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.875838128, 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.989 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:58:17.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:58:17.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:17.993 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8756380419999998, 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:58:17.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8756124700000001, 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:58:18.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8754314189999999, 2024-09-17T22:58:18.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:58:18.006 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8753959899999999, 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.007 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:58:18.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:18.011 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.875297089, 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:58:18.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:18.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:58:18.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.016 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8751209289999999, 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.017 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:58:18.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:58:18.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:18.021 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8750614329999999, 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.022 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:18.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.027 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.864788264, 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.028 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:58:18.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.032 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.864700088, 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:58:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:58:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8646471089999999, 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:58:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8645891450000001, 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:58:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:58:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.864304859, 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:58:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:58:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.864128676, 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8634495389999999, 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:58:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 1.8634260650000001, 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:58:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:18.065 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:18.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:18.067 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:18.063+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:18.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:18.166 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:18.164+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:18.269 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:18.266+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:18.312 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:18.312 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:18.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:18.339 INFO:teuthology.orchestra.run.smithi159.stdout:4654 write 0xae3c82b thru 0xae45a14 (0x91ea bytes) 2024-09-17T22:58:18.343 INFO:teuthology.orchestra.run.smithi159.stdout:4655 punch from 0x849548f to 0x84a3090, (0xdc01 bytes) 2024-09-17T22:58:18.358 INFO:teuthology.orchestra.run.smithi159.stdout:4656 trunc from 0xea33f12 to 0x2c9dc07 2024-09-17T22:58:18.368 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:18.366+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:18.372 INFO:teuthology.orchestra.run.smithi159.stdout:4657 trunc from 0x2c9dc07 to 0x43a499 2024-09-17T22:58:18.381 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:18.381 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:18.390 INFO:teuthology.orchestra.run.smithi159.stdout:4658 punch from 0x2c559f to 0x2d3433, (0xde94 bytes) 2024-09-17T22:58:18.390 INFO:teuthology.orchestra.run.smithi159.stdout:4659 punch from 0x1bddc3 to 0x1cb8fe, (0xdb3b bytes) 2024-09-17T22:58:18.391 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.406 INFO:teuthology.orchestra.run.smithi159.stdout:4662 writesame 0x7a1e693 thru 0x7a27b47 data_size 0x1(0x94b5 bytes) 2024-09-17T22:58:18.406 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.414 INFO:teuthology.orchestra.run.smithi159.stdout:4665 write 0x9c0a0bf thru 0x9c16e09 (0xcd4b bytes) 2024-09-17T22:58:18.416 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:18.416 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:18.418 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.418 INFO:teuthology.orchestra.run.smithi159.stdout:4667 write 0x4ca4e02 thru 0x4ca5a69 (0xc68 bytes) 2024-09-17T22:58:18.422 INFO:teuthology.orchestra.run.smithi159.stdout:4668 read 0x9656bb thru 0x969c8b (0x45d1 bytes) 2024-09-17T22:58:18.422 INFO:teuthology.orchestra.run.smithi159.stdout:4669 read 0x5d9c504 thru 0x5da1f44 (0x5a41 bytes) 2024-09-17T22:58:18.422 INFO:teuthology.orchestra.run.smithi159.stdout:4670 writesame 0x1da356b thru 0x1dadd2b data_size 0x23(0xa7c1 bytes) 2024-09-17T22:58:18.425 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.426 INFO:teuthology.orchestra.run.smithi159.stdout:4672 writesame 0x7caa550 thru 0x7cb30cf data_size 0x20(0x8b80 bytes) 2024-09-17T22:58:18.430 INFO:teuthology.orchestra.run.smithi159.stdout:4673 punch from 0x8386f22 to 0x838952d, (0x260b bytes) 2024-09-17T22:58:18.430 INFO:teuthology.orchestra.run.smithi159.stdout:4674 read 0x60a8cbf thru 0x60af13b (0x647d bytes) 2024-09-17T22:58:18.431 INFO:teuthology.orchestra.run.smithi159.stdout:4676 read 0x12476b4 thru 0x124f09b (0x79e8 bytes) 2024-09-17T22:58:18.431 INFO:teuthology.orchestra.run.smithi159.stdout:4677 writesame 0x2afda1c thru 0x2b022a9 data_size 0x25(0x488e bytes) 2024-09-17T22:58:18.437 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:58:18.437 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:58:18.442 INFO:teuthology.orchestra.run.smithi159.stdout:4678 write 0xab365e8 thru 0xab39c5b (0x3674 bytes) 2024-09-17T22:58:18.446 INFO:teuthology.orchestra.run.smithi159.stdout:4680 punch from 0xa7b981f to 0xa7bcaed, (0x32ce bytes) 2024-09-17T22:58:18.455 INFO:teuthology.orchestra.run.smithi159.stdout:4681 trunc from 0xab39c5c to 0x3f68611 2024-09-17T22:58:18.461 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:18.461 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:18.469 INFO:teuthology.orchestra.run.smithi159.stdout:4682 punch from 0x61cc4e to 0x6213e0, (0x4792 bytes) 2024-09-17T22:58:18.469 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:58:18.466+0000 7f6733a0e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33565) UID: 0 2024-09-17T22:58:18.471 INFO:teuthology.orchestra.run.smithi159.stdout:4683 trunc from 0x3f68611 to 0x612eff4 2024-09-17T22:58:18.472 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:18.472 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:18.483 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:18.484 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:18.485 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:18.489 INFO:teuthology.orchestra.run.smithi159.stdout:4685 writesame 0xb771c13 thru 0xb77c3bd data_size 0x1(0xa7ab bytes) 2024-09-17T22:58:18.490 INFO:teuthology.orchestra.run.smithi159.stdout:4686 write 0x66e63c7 thru 0x66ef02c (0x8c66 bytes) 2024-09-17T22:58:18.493 INFO:teuthology.orchestra.run.smithi159.stdout:4690 read 0x787a0d0 thru 0x787a809 (0x73a bytes) 2024-09-17T22:58:18.493 INFO:teuthology.orchestra.run.smithi159.stdout:4691 punch from 0x9fafad9 to 0x9fb988e, (0x9db5 bytes) 2024-09-17T22:58:18.500 INFO:teuthology.orchestra.run.smithi159.stdout:4692 write 0xc7a9669 thru 0xc7af338 (0x5cd0 bytes) 2024-09-17T22:58:18.503 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:18.503 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:4693 read 0xc14e56c thru 0xc15d428 (0xeebd bytes) 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:4695 read 0x706cef thru 0x70807e (0x1390 bytes) 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:4696 read 0x6e86e4d thru 0x6e8cf20 (0x60d4 bytes) 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:4700 punch from 0xa243ea9 to 0xa244e84, (0xfdb bytes) 2024-09-17T22:58:18.504 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.505 INFO:teuthology.orchestra.run.smithi159.stdout:4702 punch from 0x7274646 to 0x727a35c, (0x5d16 bytes) 2024-09-17T22:58:18.505 INFO:teuthology.orchestra.run.smithi159.stdout:4703 write 0x3d05fba thru 0x3d100b1 (0xa0f8 bytes) 2024-09-17T22:58:18.507 INFO:teuthology.orchestra.run.smithi159.stdout:4704 write 0x78663 thru 0x8477f (0xc11d bytes) 2024-09-17T22:58:18.510 INFO:teuthology.orchestra.run.smithi159.stdout:4705 read 0xb6750d5 thru 0xb679ee0 (0x4e0c bytes) 2024-09-17T22:58:18.512 INFO:teuthology.orchestra.run.smithi159.stdout:4706 trunc from 0xc7af339 to 0xda61842 2024-09-17T22:58:18.517 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.517 INFO:teuthology.orchestra.run.smithi159.stdout:4709 read 0x8a13b44 thru 0x8a20169 (0xc626 bytes) 2024-09-17T22:58:18.517 INFO:teuthology.orchestra.run.smithi159.stdout:4711 read 0xaf6e5aa thru 0xaf792d2 (0xad29 bytes) 2024-09-17T22:58:18.517 INFO:teuthology.orchestra.run.smithi159.stdout:4712 writesame 0xd4df579 thru 0xd4e6033 data_size 0x1(0x6abb bytes) 2024-09-17T22:58:18.521 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:18.531 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.531 INFO:teuthology.orchestra.run.smithi159.stdout:4715 read 0x886fbc5 thru 0x887ec0d (0xf049 bytes) 2024-09-17T22:58:18.538 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:18.539 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:18.545 INFO:teuthology.orchestra.run.smithi159.stdout:4717 trunc from 0xda61842 to 0x358d399 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:4719 writesame 0xb39315a thru 0xb3a2201 data_size 0x22(0xf0a8 bytes) 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:4724 read 0x9a6d3d3 thru 0x9a75d58 (0x8986 bytes) 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:4727 punch from 0x78bffb to 0x78d151, (0x1156 bytes) 2024-09-17T22:58:18.574 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.574 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:18.571+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:18.575 INFO:teuthology.orchestra.run.smithi159.stdout:4729 trunc from 0xb3a2202 to 0xbbd9037 2024-09-17T22:58:18.580 INFO:teuthology.orchestra.run.smithi159.stdout:4730 writesame 0xae5a8b3 thru 0xae68ebc data_size 0x1(0xe60a bytes) 2024-09-17T22:58:18.598 INFO:teuthology.orchestra.run.smithi159.stdout:4731 read 0x33fac69 thru 0x33ff757 (0x4aef bytes) 2024-09-17T22:58:18.598 INFO:teuthology.orchestra.run.smithi159.stdout:4733 read 0xb1c39b2 thru 0xb1cff95 (0xc5e4 bytes) 2024-09-17T22:58:18.598 INFO:teuthology.orchestra.run.smithi159.stdout:4734 punch from 0x22b967 to 0x232c48, (0x72e1 bytes) 2024-09-17T22:58:18.598 INFO:teuthology.orchestra.run.smithi159.stdout:4735 read 0x3335bea thru 0x3335bf9 (0x10 bytes) 2024-09-17T22:58:18.599 INFO:teuthology.orchestra.run.smithi159.stdout:4736 trunc from 0xbbd9037 to 0xd2badee 2024-09-17T22:58:18.606 INFO:teuthology.orchestra.run.smithi159.stdout:4738 read 0xb26d67b thru 0xb276faa (0x9930 bytes) 2024-09-17T22:58:18.606 INFO:teuthology.orchestra.run.smithi159.stdout:4739 read 0x353c94f thru 0x353d0ea (0x79c bytes) 2024-09-17T22:58:18.606 INFO:teuthology.orchestra.run.smithi159.stdout:4740 read 0x4d6b177 thru 0x4d715a6 (0x6430 bytes) 2024-09-17T22:58:18.606 INFO:teuthology.orchestra.run.smithi159.stdout:4743 write 0x62b6c05 thru 0x62ba9b4 (0x3db0 bytes) 2024-09-17T22:58:18.611 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.611 INFO:teuthology.orchestra.run.smithi159.stdout:4745 write 0x6d84cb3 thru 0x6d8a14b (0x5499 bytes) 2024-09-17T22:58:18.618 INFO:teuthology.orchestra.run.smithi159.stdout:4746 punch from 0x810db7e to 0x8111b2e, (0x3fb0 bytes) 2024-09-17T22:58:18.618 INFO:teuthology.orchestra.run.smithi159.stdout:4747 read 0xb7cf2de thru 0xb7d0ec7 (0x1bea bytes) 2024-09-17T22:58:18.618 INFO:teuthology.orchestra.run.smithi159.stdout:4748 write 0xa1ed81a thru 0xa1f7f3a (0xa721 bytes) 2024-09-17T22:58:18.619 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:18.619 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:18.622 INFO:teuthology.orchestra.run.smithi159.stdout:4750 writesame 0xbdc8794 thru 0xbdd55cc data_size 0x1f(0xce39 bytes) 2024-09-17T22:58:18.627 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.627 INFO:teuthology.orchestra.run.smithi159.stdout:4752 read 0xcef221a thru 0xcef3a96 (0x187d bytes) 2024-09-17T22:58:18.627 INFO:teuthology.orchestra.run.smithi159.stdout:4753 read 0x46f95e2 thru 0x4704aed (0xb50c bytes) 2024-09-17T22:58:18.627 INFO:teuthology.orchestra.run.smithi159.stdout:4755 read 0x9343a21 thru 0x934eade (0xb0be bytes) 2024-09-17T22:58:18.627 INFO:teuthology.orchestra.run.smithi159.stdout:4757 writesame 0x3e82a96 thru 0x3e86d81 data_size 0x2(0x42ec bytes) 2024-09-17T22:58:18.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:58:18.634 INFO:teuthology.orchestra.run.smithi159.stdout:4758 write 0x34fa5ab thru 0x34fe8d9 (0x432f bytes) 2024-09-17T22:58:18.637 INFO:teuthology.orchestra.run.smithi159.stdout:4759 write 0x4b57b22 thru 0x4b62745 (0xac24 bytes) 2024-09-17T22:58:18.641 INFO:teuthology.orchestra.run.smithi159.stdout:4761 write 0x29737f3 thru 0x297b9db (0x81e9 bytes) 2024-09-17T22:58:18.642 INFO:teuthology.orchestra.run.smithi159.stdout:4762 read 0x639954 thru 0x643e68 (0xa515 bytes) 2024-09-17T22:58:18.643 INFO:teuthology.orchestra.run.smithi159.stdout:4763 write 0x48a8df4 thru 0x48aa0cf (0x12dc bytes) 2024-09-17T22:58:18.644 INFO:teuthology.orchestra.run.smithi159.stdout:4764 write 0x877122 thru 0x8859f4 (0xe8d3 bytes) 2024-09-17T22:58:18.648 INFO:teuthology.orchestra.run.smithi159.stdout:4766 read 0x83ef02b thru 0x83fda30 (0xea06 bytes) 2024-09-17T22:58:18.648 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.649 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.649 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.649 INFO:teuthology.orchestra.run.smithi159.stdout:4771 writesame 0x69a9c4a thru 0x69ab397 data_size 0x13(0x174e bytes) 2024-09-17T22:58:18.651 INFO:teuthology.orchestra.run.smithi159.stdout:4772 write 0x849f81d thru 0x84a2d74 (0x3558 bytes) 2024-09-17T22:58:18.656 INFO:teuthology.orchestra.run.smithi159.stdout:4773 read 0x31f2b56 thru 0x31f731c (0x47c7 bytes) 2024-09-17T22:58:18.656 INFO:teuthology.orchestra.run.smithi159.stdout:4775 read 0x3fb4fbb thru 0x3fc0a5d (0xbaa3 bytes) 2024-09-17T22:58:18.656 INFO:teuthology.orchestra.run.smithi159.stdout:4777 punch from 0x98889e8 to 0x988d2ed, (0x4905 bytes) 2024-09-17T22:58:18.656 INFO:teuthology.orchestra.run.smithi159.stdout:4778 read 0xa2b8b82 thru 0xa2bca82 (0x3f01 bytes) 2024-09-17T22:58:18.657 INFO:teuthology.orchestra.run.smithi159.stdout:4779 read 0x20a12b9 thru 0x20b054e (0xf296 bytes) 2024-09-17T22:58:18.657 INFO:teuthology.orchestra.run.smithi159.stdout:4780 read 0x334d9ba thru 0x335c592 (0xebd9 bytes) 2024-09-17T22:58:18.662 INFO:teuthology.orchestra.run.smithi159.stdout:4781 trunc from 0xd2badee to 0x8b74dff 2024-09-17T22:58:18.671 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:18.668+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:18.673 INFO:teuthology.orchestra.run.smithi159.stdout:4782 read 0x370092a thru 0x370bf93 (0xb66a bytes) 2024-09-17T22:58:18.673 INFO:teuthology.orchestra.run.smithi159.stdout:4783 read 0x242ea21 thru 0x24387ed (0x9dcd bytes) 2024-09-17T22:58:18.673 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.674 INFO:teuthology.orchestra.run.smithi159.stdout:4786 punch from 0x3edf4fb to 0x3eee126, (0xec2b bytes) 2024-09-17T22:58:18.674 INFO:teuthology.orchestra.run.smithi159.stdout:4787 punch from 0x1b8f51f to 0x1b94e4c, (0x592d bytes) 2024-09-17T22:58:18.674 INFO:teuthology.orchestra.run.smithi159.stdout:4788 trunc from 0x8b74dff to 0x8f12e81 2024-09-17T22:58:18.678 INFO:teuthology.orchestra.run.smithi159.stdout:4789 read 0x62a0876 thru 0x62af2e9 (0xea74 bytes) 2024-09-17T22:58:18.679 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.691 INFO:teuthology.orchestra.run.smithi159.stdout:4791 write 0xe7b717c thru 0xe7c4fba (0xde3f bytes) 2024-09-17T22:58:18.695 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:18.695 INFO:teuthology.orchestra.run.smithi159.stdout:4793 read 0x8875318 thru 0x8883458 (0xe141 bytes) 2024-09-17T22:58:18.695 INFO:teuthology.orchestra.run.smithi159.stdout:4795 write 0x81f73c9 thru 0x81f8895 (0x14cd bytes) 2024-09-17T22:58:18.699 INFO:teuthology.orchestra.run.smithi159.stdout:4796 punch from 0x8f79f5a to 0x8f86b23, (0xcbc9 bytes) 2024-09-17T22:58:18.699 INFO:teuthology.orchestra.run.smithi159.stdout:4797 read 0x7c17a3 thru 0x7cbd53 (0xa5b1 bytes) 2024-09-17T22:58:18.699 INFO:teuthology.orchestra.run.smithi159.stdout:4798 punch from 0x66ddb44 to 0x66e30ef, (0x55ab bytes) 2024-09-17T22:58:18.699 INFO:teuthology.orchestra.run.smithi159.stdout:4799 punch from 0x3a741da to 0x3a826e3, (0xe509 bytes) 2024-09-17T22:58:18.699 INFO:teuthology.orchestra.run.smithi159.stdout:4800 writesame 0xe11c74a thru 0xe12a620 data_size 0x1(0xded7 bytes) 2024-09-17T22:58:18.717 INFO:teuthology.orchestra.run.smithi159.stdout:4803 write 0xd1be414 thru 0xd1caa66 (0xc653 bytes) 2024-09-17T22:58:18.732 INFO:teuthology.orchestra.run.smithi159.stdout:4807 trunc from 0xe7c4fbb to 0x4d293c3 2024-09-17T22:58:18.744 INFO:teuthology.orchestra.run.smithi159.stdout:4808 clone 35 order 23 su 65536 sc 7 2024-09-17T22:58:18.751 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:18.751 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:18.760 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:58:18.772 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:18.770+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:18.873 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:18.871+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:18.904 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:18.904 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:18.913 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:58:18.957 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:58:18.974 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:18.971+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:19.024 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.024 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.035 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:19.075 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:19.072+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:19.078 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T22:58:19.078 INFO:teuthology.orchestra.run.smithi159.stdout:4809 read 0x3055b04 thru 0x3058da5 (0x32a2 bytes) 2024-09-17T22:58:19.088 INFO:teuthology.orchestra.run.smithi159.stdout:4814 write 0x65cae8e thru 0x65d0f61 (0x60d4 bytes) 2024-09-17T22:58:19.098 INFO:teuthology.orchestra.run.smithi159.stdout:4815 read 0x211f43e thru 0x212299c (0x355f bytes) 2024-09-17T22:58:19.099 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:19.103 INFO:teuthology.orchestra.run.smithi159.stdout:4818 trunc from 0x65d0f62 to 0x1d03dd4 2024-09-17T22:58:19.176 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:19.174+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:19.182 INFO:teuthology.orchestra.run.smithi159.stdout:4819 writesame 0xe41980a thru 0xe421f28 data_size 0x1(0x871f bytes) 2024-09-17T22:58:19.182 INFO:teuthology.orchestra.run.smithi159.stdout:4821 punch from 0x10c243 to 0x10e92e, (0x26eb bytes) 2024-09-17T22:58:19.182 INFO:teuthology.orchestra.run.smithi159.stdout:4823 punch from 0x5618e57 to 0x5619fe9, (0x1192 bytes) 2024-09-17T22:58:19.182 INFO:teuthology.orchestra.run.smithi159.stdout:4826 writesame 0x70bd898 thru 0x70c7b84 data_size 0x1(0xa2ed bytes) 2024-09-17T22:58:19.203 INFO:teuthology.orchestra.run.smithi159.stdout:4827 write 0x5405b68 thru 0x540f9cd (0x9e66 bytes) 2024-09-17T22:58:19.204 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.204 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.219 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:19.220 INFO:teuthology.orchestra.run.smithi159.stdout:4828 trunc from 0xe421f29 to 0x30fbb53 2024-09-17T22:58:19.232 INFO:teuthology.orchestra.run.smithi159.stdout:4831 clone 36 order 20 su 4096 sc 2 2024-09-17T22:58:19.277 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:19.274+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:19.323 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.324 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:19.417 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:19.415+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.477 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:19.475+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:19.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:19.528 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-17T22:58:19.528 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:19.529 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-17T22:58:19.529 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T22:58:19.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:58:19.557 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T22:58:19.557 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-17T22:58:19.557 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T22:58:19.557 DEBUG:teuthology.orchestra.run.smithi159:> type ceph-objectstore-tool 2024-09-17T22:58:19.578 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:19.575+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:19.589 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.589 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.598 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:19.612 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:58:19.613 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-17T22:58:19.696 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:19.694+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:19.742 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.742 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:19.780 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:19.778+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:19.853 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.853 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.862 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:19.906 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:19.904+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:19.967 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:19.967 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:19.976 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:20.017 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:20.015+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:20.082 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:20.080+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:20.120 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:20.120 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:20.128 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:20.184 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:20.181+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:20.272 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:20.273 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:20.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:20.284 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:20.282+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:20.385 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:20.385 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:20.385 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:20.383+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:20.393 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:20.486 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:20.484+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:20.496 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:20.497 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:20.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:20.554 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:20.598 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:20.596+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:20.609 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:20.609 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:20.618 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:20.688 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:20.687+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:20.699 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:20.699 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:20.716 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:20.717 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:20.717 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:20.732 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:20.733 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:20.743 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:20.743 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:20.743 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:20.743 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:20.749 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:20.749 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:20.752 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:20.766 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:20.766 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:20.783 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:20.783 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:20.789 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:20.787+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:20.799 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:20.799 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:20.802 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:20.803 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:20.856 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:20.856 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:20.856 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:20.856 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:20.856 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:20.864 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:20.894 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:20.892+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:20.971 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:58:20.975 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.705649889, 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.705310875000002, 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 36.52767188, 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8834816329999997, 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8834590039999997, 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8832589180000001, 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:58:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8832333459999999, 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:58:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8830522949999997, 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:58:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:58:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8830168660000002, 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:58:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8829179649999999, 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8827418050000002, 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:58:21.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8826823089999998, 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:58:21.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:58:21.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8724091400000002, 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:58:21.032 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.036 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.872320964, 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:21.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8722679849999997, 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:58:21.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.046 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8722100209999999, 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:58:21.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:58:21.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.051 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8719257349999996, 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:21.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:21.056 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8717495519999998, 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:58:21.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8710704150000002, 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.061 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.8710469410000004, 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:58:21.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.070 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.072 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:21.072 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone35 from 0x30fbb53 (overlap 0x1d03dd4) to 0x0 2024-09-17T22:58:21.072 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T22:58:21.072 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:21.069+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:21.160 INFO:teuthology.orchestra.run.smithi159.stdout:4832 punch from 0x287bd3d to 0x2887bf9, (0xbebc bytes) 2024-09-17T22:58:21.164 INFO:teuthology.orchestra.run.smithi159.stdout:4835 read 0x147d897 thru 0x148184b (0x3fb5 bytes) 2024-09-17T22:58:21.172 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:21.170+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:21.184 INFO:teuthology.orchestra.run.smithi159.stdout:4836 writesame 0xe73bdd8 thru 0xe741133 data_size 0x61(0x535c bytes) 2024-09-17T22:58:21.184 INFO:teuthology.orchestra.run.smithi159.stdout:4837 punch from 0x4aa8851 to 0x4ab11c5, (0x8974 bytes) 2024-09-17T22:58:21.184 INFO:teuthology.orchestra.run.smithi159.stdout:4838 read 0xc3689f0 thru 0xc3716dd (0x8cee bytes) 2024-09-17T22:58:21.184 INFO:teuthology.orchestra.run.smithi159.stdout:4839 write 0x1d67b9d thru 0x1d6e888 (0x6cec bytes) 2024-09-17T22:58:21.215 INFO:teuthology.orchestra.run.smithi159.stdout:4841 write 0xe0a80ef thru 0xe0b603d (0xdf4f bytes) 2024-09-17T22:58:21.219 INFO:teuthology.orchestra.run.smithi159.stdout:4843 write 0x3d457e2 thru 0x3d45e01 (0x620 bytes) 2024-09-17T22:58:21.222 INFO:teuthology.orchestra.run.smithi159.stdout:4844 write 0xdc31bab thru 0xdc3ceec (0xb342 bytes) 2024-09-17T22:58:21.226 INFO:teuthology.orchestra.run.smithi159.stdout:4846 punch from 0x78ea6d7 to 0x78ed08a, (0x29b3 bytes) 2024-09-17T22:58:21.226 INFO:teuthology.orchestra.run.smithi159.stdout:4847 writesame 0x6dc362b thru 0x6dcfeda data_size 0x98(0xc8b0 bytes) 2024-09-17T22:58:21.230 INFO:teuthology.orchestra.run.smithi159.stdout:4849 read 0x3c57b21 thru 0x3c5cc08 (0x50e8 bytes) 2024-09-17T22:58:21.230 INFO:teuthology.orchestra.run.smithi159.stdout:4851 punch from 0x9cd758 to 0x9cef02, (0x17aa bytes) 2024-09-17T22:58:21.230 INFO:teuthology.orchestra.run.smithi159.stdout:4852 punch from 0xc291a23 to 0xc29224b, (0x828 bytes) 2024-09-17T22:58:21.230 INFO:teuthology.orchestra.run.smithi159.stdout:4853 write 0x6425507 thru 0x6425ba5 (0x69f bytes) 2024-09-17T22:58:21.233 INFO:teuthology.orchestra.run.smithi159.stdout:4854 read 0x98385ea thru 0x984500c (0xca23 bytes) 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout:4855 read 0x846e980 thru 0x8475df5 (0x7476 bytes) 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout:4856 write 0x3ec2f3f thru 0x3ecaa70 (0x7b32 bytes) 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:21.234 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.237 INFO:teuthology.orchestra.run.smithi159.stdout:4857 write 0x41b279a thru 0x41ba85c (0x80c3 bytes) 2024-09-17T22:58:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:4858 write 0xc12f432 thru 0xc135d6f (0x693e bytes) 2024-09-17T22:58:21.244 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:21.245 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.245 INFO:teuthology.orchestra.run.smithi159.stdout:4861 punch from 0x39f00bc to 0x39fc1e4, (0xc128 bytes) 2024-09-17T22:58:21.245 INFO:teuthology.orchestra.run.smithi159.stdout:4863 read 0x6f91390 thru 0x6f99323 (0x7f94 bytes) 2024-09-17T22:58:21.250 INFO:teuthology.orchestra.run.smithi159.stdout:4864 trunc from 0xe741134 to 0x98a8439 2024-09-17T22:58:21.267 INFO:teuthology.orchestra.run.smithi159.stdout:4865 writesame 0xa3bd538 thru 0xa3bdd08 data_size 0x17(0x7d1 bytes) 2024-09-17T22:58:21.268 INFO:teuthology.orchestra.run.smithi159.stdout:4867 punch from 0x2824feb to 0x28304b5, (0xb4ca bytes) 2024-09-17T22:58:21.268 INFO:teuthology.orchestra.run.smithi159.stdout:4868 read 0xcdb3c3 thru 0xce147e (0x60bc bytes) 2024-09-17T22:58:21.269 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.270 INFO:teuthology.orchestra.run.smithi159.stdout:4870 read 0x88f2f04 thru 0x8900c8f (0xdd8c bytes) 2024-09-17T22:58:21.270 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.270 INFO:teuthology.orchestra.run.smithi159.stdout:4874 write 0x3e93f4d thru 0x3e9d750 (0x9804 bytes) 2024-09-17T22:58:21.271 INFO:teuthology.orchestra.run.smithi159.stdout:4877 punch from 0x2481fb7 to 0x248260e, (0x657 bytes) 2024-09-17T22:58:21.271 INFO:teuthology.orchestra.run.smithi159.stdout:4878 trunc from 0xa3bdd09 to 0xa88126d 2024-09-17T22:58:21.273 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:21.271+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:21.275 INFO:teuthology.orchestra.run.smithi159.stdout:4879 read 0x953e5ee thru 0x9547e89 (0x989c bytes) 2024-09-17T22:58:21.275 INFO:teuthology.orchestra.run.smithi159.stdout:4880 read 0xb61dba thru 0xb647aa (0x29f1 bytes) 2024-09-17T22:58:21.276 INFO:teuthology.orchestra.run.smithi159.stdout:4881 trunc from 0xa88126d to 0xa4359a7 2024-09-17T22:58:21.289 INFO:teuthology.orchestra.run.smithi159.stdout:4882 writesame 0xbb01433 thru 0xbb0e979 data_size 0x1(0xd547 bytes) 2024-09-17T22:58:21.289 INFO:teuthology.orchestra.run.smithi159.stdout:4883 punch from 0x8770982 to 0x8779845, (0x8ec3 bytes) 2024-09-17T22:58:21.289 INFO:teuthology.orchestra.run.smithi159.stdout:4884 read 0x77a54c thru 0x789894 (0xf349 bytes) 2024-09-17T22:58:21.291 INFO:teuthology.orchestra.run.smithi159.stdout:4885 write 0x3ce5cb2 thru 0x3cf0341 (0xa690 bytes) 2024-09-17T22:58:21.294 INFO:teuthology.orchestra.run.smithi159.stdout:4888 read 0x3011436 thru 0x301ef1f (0xdaea bytes) 2024-09-17T22:58:21.301 INFO:teuthology.orchestra.run.smithi159.stdout:4890 punch from 0x1eeb594 to 0x1eee4cc, (0x2f38 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4892 punch from 0x3436d64 to 0x343b429, (0x46c5 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4893 punch from 0x57b3cc8 to 0x57b7b91, (0x3ec9 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4894 punch from 0xf08993 to 0xf090a6, (0x713 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4896 read 0x81395b5 thru 0x813c889 (0x32d5 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4897 punch from 0x20f9ac7 to 0x2106cbd, (0xd1f6 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4898 read 0xa9b5bf1 thru 0xa9bc9f5 (0x6e05 bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4899 read 0x7279dd2 thru 0x72898cb (0xfafa bytes) 2024-09-17T22:58:21.302 INFO:teuthology.orchestra.run.smithi159.stdout:4900 read 0x1edb5fe thru 0x1ee6c54 (0xb657 bytes) 2024-09-17T22:58:21.303 INFO:teuthology.orchestra.run.smithi159.stdout:4901 read 0x2f7238a thru 0x2f7e579 (0xc1f0 bytes) 2024-09-17T22:58:21.303 INFO:teuthology.orchestra.run.smithi159.stdout:4902 punch from 0x9158767 to 0x915e8be, (0x6157 bytes) 2024-09-17T22:58:21.303 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.303 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.307 INFO:teuthology.orchestra.run.smithi159.stdout:4905 trunc from 0xbb0e97a to 0x5aa5197 2024-09-17T22:58:21.325 INFO:teuthology.orchestra.run.smithi159.stdout:4907 trunc from 0x5aa5197 to 0xb5c46c7 2024-09-17T22:58:21.343 INFO:teuthology.orchestra.run.smithi159.stdout:4908 trunc from 0xb5c46c7 to 0x59c57d 2024-09-17T22:58:21.375 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:21.373+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:21.383 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:21.383 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:21.383 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:21.383 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:21.383 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.386 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.392 INFO:teuthology.orchestra.run.smithi159.stdout:4910 write 0x13642aa thru 0x136fdce (0xbb25 bytes) 2024-09-17T22:58:21.394 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:21.396 INFO:teuthology.orchestra.run.smithi159.stdout:4911 punch from 0x10647b5 to 0x1070448, (0xbc93 bytes) 2024-09-17T22:58:21.396 INFO:teuthology.orchestra.run.smithi159.stdout:4913 read 0xe5fb2b thru 0xe68185 (0x865b bytes) 2024-09-17T22:58:21.408 INFO:teuthology.orchestra.run.smithi159.stdout:4916 trunc from 0x136fdcf to 0xb1bec23 2024-09-17T22:58:21.414 INFO:teuthology.orchestra.run.smithi159.stdout:4918 write 0x878ccdd thru 0x8794a0e (0x7d32 bytes) 2024-09-17T22:58:21.419 INFO:teuthology.orchestra.run.smithi159.stdout:4919 write 0x48a7993 thru 0x48b001c (0x868a bytes) 2024-09-17T22:58:21.424 INFO:teuthology.orchestra.run.smithi159.stdout:4920 read 0x2685e13 thru 0x2687341 (0x152f bytes) 2024-09-17T22:58:21.424 INFO:teuthology.orchestra.run.smithi159.stdout:4921 punch from 0x3aa03f8 to 0x3aad419, (0xd021 bytes) 2024-09-17T22:58:21.424 INFO:teuthology.orchestra.run.smithi159.stdout:4922 writesame 0xa5e05b1 thru 0xa5e9309 data_size 0x1(0x8d59 bytes) 2024-09-17T22:58:21.436 INFO:teuthology.orchestra.run.smithi159.stdout:4923 writesame 0x762996e thru 0x7630137 data_size 0x5(0x67ca bytes) 2024-09-17T22:58:21.442 INFO:teuthology.orchestra.run.smithi159.stdout:4924 write 0x3d5d513 thru 0x3d6b6f6 (0xe1e4 bytes) 2024-09-17T22:58:21.445 INFO:teuthology.orchestra.run.smithi159.stdout:4927 read 0x810430a thru 0x81085a4 (0x429b bytes) 2024-09-17T22:58:21.448 INFO:teuthology.orchestra.run.smithi159.stdout:4928 trunc from 0xb1bec23 to 0x90f017c 2024-09-17T22:58:21.459 INFO:teuthology.orchestra.run.smithi159.stdout:4930 read 0x7f1362d thru 0x7f21717 (0xe0eb bytes) 2024-09-17T22:58:21.459 INFO:teuthology.orchestra.run.smithi159.stdout:4931 read 0x45f5f36 thru 0x45faac2 (0x4b8d bytes) 2024-09-17T22:58:21.459 INFO:teuthology.orchestra.run.smithi159.stdout:4932 read 0x7a6d1af thru 0x7a74747 (0x7599 bytes) 2024-09-17T22:58:21.459 INFO:teuthology.orchestra.run.smithi159.stdout:4933 punch from 0x61a3843 to 0x61a48ed, (0x10aa bytes) 2024-09-17T22:58:21.459 INFO:teuthology.orchestra.run.smithi159.stdout:4934 read 0x53d931 thru 0x540e62 (0x3532 bytes) 2024-09-17T22:58:21.465 INFO:teuthology.orchestra.run.smithi159.stdout:4936 write 0x9d82923 thru 0x9d90d3a (0xe418 bytes) 2024-09-17T22:58:21.470 INFO:teuthology.orchestra.run.smithi159.stdout:4937 read 0x25490da thru 0x25523bd (0x92e4 bytes) 2024-09-17T22:58:21.470 INFO:teuthology.orchestra.run.smithi159.stdout:4938 read 0x4750a56 thru 0x475cd5e (0xc309 bytes) 2024-09-17T22:58:21.470 INFO:teuthology.orchestra.run.smithi159.stdout:4939 read 0x259e4c3 thru 0x25ab592 (0xd0d0 bytes) 2024-09-17T22:58:21.476 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:21.474+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:21.479 INFO:teuthology.orchestra.run.smithi159.stdout:4940 trunc from 0x9d90d3b to 0x1bf3694 2024-09-17T22:58:21.507 INFO:teuthology.orchestra.run.smithi159.stdout:4942 write 0xb86e9d7 thru 0xb876887 (0x7eb1 bytes) 2024-09-17T22:58:21.516 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4746226240000002, 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4746022090000004, 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4745811499999997, 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4745580450000002, 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.540 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:21.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4745349890000004, 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.545 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:21.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4745043559999997, 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:21.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:21.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:21.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4744803800000001, 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:21.555 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:21.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:21.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4744502019999999, 2024-09-17T22:58:21.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.564 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.474431225, 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.565 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:21.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:21.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.569 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4739923140000002, 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4737037089999996, 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.574 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:21.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4730768049999998, 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:21.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4730423659999996, 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:21.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.587 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4726542059999996, 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:21.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:21.592 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4708740740000001, 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:21.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:21.597 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.470859806, 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.598 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:21.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4708493499999999, 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4708310689999999, 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.608 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:21.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:21.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.613 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4704116010000003, 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.614 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:21.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:21.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.469785356, 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:21.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:21.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:21.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:21.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:21.623 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.624 INFO:teuthology.orchestra.run.smithi159.stdout:4943 trunc from 0xb876888 to 0x27e974d 2024-09-17T22:58:21.625 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:21.626 INFO:teuthology.orchestra.run.smithi159.stdout:4945 trunc from 0x27e974d to 0x58048ad 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4948 write 0x1390aa thru 0x139230 (0x187 bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4949 writesame 0x4c12b5f thru 0x4c14a35 data_size 0x1(0x1ed7 bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4950 trunc from 0x58048ad to 0x8ba6da 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4951 writesame 0x4f9da07 thru 0x4fa06e8 data_size 0x6(0x2ce2 bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4952 read 0x359a85a thru 0x35a59ff (0xb1a6 bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4955 punch from 0x1d59970 to 0x1d61dba, (0x844a bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:4956 write 0xf9ec7d thru 0xfadd67 (0xf0eb bytes) 2024-09-17T22:58:21.627 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:21.628 INFO:teuthology.orchestra.run.smithi159.stdout:4959 write 0x947b523 thru 0x9488983 (0xd461 bytes) 2024-09-17T22:58:21.628 INFO:teuthology.orchestra.run.smithi159.stdout:4960 trunc from 0x9488984 to 0x2f6016e 2024-09-17T22:58:21.628 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:21.622+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:21.631 INFO:teuthology.orchestra.run.smithi159.stdout:4962 read 0x2f1a992 thru 0x2f250ea (0xa759 bytes) 2024-09-17T22:58:21.644 INFO:teuthology.orchestra.run.smithi159.stdout:4963 trunc from 0x2f6016e to 0xd786885 2024-09-17T22:58:21.649 INFO:teuthology.orchestra.run.smithi159.stdout:4964 read 0xa1389fe thru 0xa139d74 (0x1377 bytes) 2024-09-17T22:58:21.649 INFO:teuthology.orchestra.run.smithi159.stdout:4965 write 0x5156951 thru 0x515ee3c (0x84ec bytes) 2024-09-17T22:58:21.653 INFO:teuthology.orchestra.run.smithi159.stdout:4966 write 0x824e9ca thru 0x825697d (0x7fb4 bytes) 2024-09-17T22:58:21.654 INFO:teuthology.orchestra.run.smithi159.stdout:3.4 2024-09-17T22:58:21.654 INFO:teuthology.orchestra.run.smithi159.stdout:3.1f 2024-09-17T22:58:21.655 INFO:teuthology.orchestra.run.smithi159.stdout:3.5 2024-09-17T22:58:21.655 INFO:teuthology.orchestra.run.smithi159.stdout:2.3 2024-09-17T22:58:21.655 INFO:teuthology.orchestra.run.smithi159.stdout:3.0 2024-09-17T22:58:21.655 INFO:teuthology.orchestra.run.smithi159.stdout:2.1 2024-09-17T22:58:21.658 INFO:teuthology.orchestra.run.smithi159.stdout:4968 read 0xc0a61f8 thru 0xc0a6a8a (0x893 bytes) 2024-09-17T22:58:21.658 INFO:teuthology.orchestra.run.smithi159.stdout:4969 clone 37 order 19 su 8192 sc 3 2024-09-17T22:58:21.726 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:21.724+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:21.738 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:21.738 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:21.738 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.746 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:21.827 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:21.825+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:21.890 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:21.890 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:21.890 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:21.898 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:21.942 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:21.940+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:21.965 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 22.853560423000001, 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5017720239999992, 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.022 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5017513139999998, 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.023 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:22.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:22.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5017315189999998, 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:22.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5017122759999992, 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:22.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5016911359999998, 2024-09-17T22:58:22.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:22.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:22.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5016660470000005, 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:22.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.046 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5016467040000006, 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:22.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:22.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5016246809999991, 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:22.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5015305399999992, 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:22.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:22.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5015037679999992, 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:22.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5014832420000008, 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.063 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:22.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.066 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5014619489999994, 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:22.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:22.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.070 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5014330739999995, 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.071 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:22.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.074 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5013919950000005, 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:22.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.078 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.5013679720000006, 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:22.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.082 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.6166253450000001, 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.083 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.6166037170000003, 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.087 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:22.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.6165825329999999, 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.092 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:22.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:22.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:22.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.6165614530000001, 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:22.096 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:22.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:22.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:22.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:22.100 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:22.102 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:22.099+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:22.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:22.103 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone36 from 0xd786885 (overlap 0x59c57d) to 0x8fa26 2024-09-17T22:58:22.153 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T22:58:22.154 INFO:teuthology.orchestra.run.smithi159.stdout:4970 read 0x9eb3aef thru 0x9eb467e (0xb90 bytes) 2024-09-17T22:58:22.155 INFO:teuthology.orchestra.run.smithi159.stdout:4972 trunc from 0xd786885 to 0xcd5f278 2024-09-17T22:58:22.168 INFO:teuthology.orchestra.run.smithi159.stdout:4973 read 0x52258 thru 0x5d7df (0xb588 bytes) 2024-09-17T22:58:22.170 INFO:teuthology.orchestra.run.smithi159.stdout:4974 punch from 0x40fd90a to 0x41027d5, (0x4ecb bytes) 2024-09-17T22:58:22.170 INFO:teuthology.orchestra.run.smithi159.stdout:4975 read 0x1e643fc thru 0x1e6af98 (0x6b9d bytes) 2024-09-17T22:58:22.171 INFO:teuthology.orchestra.run.smithi159.stdout:4976 trunc from 0xcd5f278 to 0xc080feb 2024-09-17T22:58:22.175 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 2.3 --file /var/log/ceph/exp.2.3.2 2024-09-17T22:58:22.186 INFO:teuthology.orchestra.run.smithi159.stdout:4980 read 0x56659ab thru 0x5666c17 (0x126d bytes) 2024-09-17T22:58:22.194 INFO:teuthology.orchestra.run.smithi159.stdout:4981 writesame 0xdae77db thru 0xdaef24f data_size 0x1d(0x7a75 bytes) 2024-09-17T22:58:22.194 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:22.194 INFO:teuthology.orchestra.run.smithi159.stdout:4984 writesame 0x69e1cd9 thru 0x69f1bb6 data_size 0x65(0xfede bytes) 2024-09-17T22:58:22.202 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:22.200+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:22.212 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:22.212 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:22.213 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:22.213 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:22.214 INFO:teuthology.orchestra.run.smithi159.stdout:4985 trunc from 0xdaef250 to 0x5ef5bb 2024-09-17T22:58:22.222 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:22.286 INFO:teuthology.orchestra.run.smithi159.stdout:4986 writesame 0xdb443c3 thru 0xdb50bcd data_size 0x1(0xc80b bytes) 2024-09-17T22:58:22.286 INFO:teuthology.orchestra.run.smithi159.stdout:4987 write 0x1af4e1d thru 0x1af7f7b (0x315f bytes) 2024-09-17T22:58:22.290 INFO:teuthology.orchestra.run.smithi159.stdout:4989 punch from 0xb64997e to 0xb64a75f, (0xde1 bytes) 2024-09-17T22:58:22.297 INFO:teuthology.orchestra.run.smithi159.stdout:4990 trunc from 0xdb50bce to 0x7048f90 2024-09-17T22:58:22.302 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:22.300+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:22.303 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:22.303 INFO:teuthology.orchestra.run.smithi159.stdout:4992 read 0x245d535 thru 0x2468556 (0xb022 bytes) 2024-09-17T22:58:22.315 INFO:teuthology.orchestra.run.smithi159.stdout:4993 write 0xc4607eb thru 0xc466924 (0x613a bytes) 2024-09-17T22:58:22.319 INFO:teuthology.orchestra.run.smithi159.stdout:4994 write 0x4483a3c thru 0x4489d83 (0x6348 bytes) 2024-09-17T22:58:22.323 INFO:teuthology.orchestra.run.smithi159.stdout:4995 punch from 0x79759ba to 0x79781c0, (0x2806 bytes) 2024-09-17T22:58:22.324 INFO:teuthology.orchestra.run.smithi159.stdout:4997 clone 38 order 18 su 16384 sc 8 2024-09-17T22:58:22.330 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:22.330 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:22.330 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:22.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:22.423 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:22.421+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:22.484 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:22.484 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:22.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:22.484 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:22.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89486 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0957219470000004, 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059383219000000001, 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89486 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:22.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385240+0000", 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385245+0000", 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.639e-06 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385248+0000", 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1319999999999998e-06 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444094+0000", 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058845916999999998 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444207+0000", 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113261 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444621+0000", 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041399299999999998 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89487 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0957075170000001, 2024-09-17T22:58:22.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059624093000000003, 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89487 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.15e-07 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385254+0000", 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2389999999999999e-06 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385257+0000", 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385260+0000", 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.785e-06 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444667+0000", 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059407142000000003 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444750+0000", 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2232999999999997e-05 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444877+0000", 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012703399999999999 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89488 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 257024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0956967119999996, 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059823605000000002, 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89488 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385264+0000", 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385265+0000", 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.206e-06 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385268+0000", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385271+0000", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7389999999999999e-06 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444898+0000", 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059626894999999999 2024-09-17T22:58:22.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444967+0000", 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9573999999999995e-05 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445087+0000", 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011978500000000001 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0956863800000001, 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91012 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0569922429999998, 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059095525000000003, 2024-09-17T22:58:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91012 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423970+0000", 2024-09-17T22:58:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423982+0000", 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423988+0000", 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482905+0000", 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058917737999999997 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482969+0000", 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4054999999999996e-05 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483063+0000", 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3968999999999994e-05 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91013 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0569620569999998, 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059240328000000002, 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91013 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424000+0000", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424012+0000", 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1525000000000001e-05 2024-09-17T22:58:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424018+0000", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483090+0000", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059072066999999999 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483131+0000", 2024-09-17T22:58:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1427000000000002e-05 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483238+0000", 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107161 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91014 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.056935374, 2024-09-17T22:58:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059357291999999999, 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91014 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:58:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424027+0000", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424035+0000", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6389999999999992e-06 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424040+0000", 2024-09-17T22:58:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9389999999999996e-06 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483266+0000", 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059226144000000001 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483326+0000", 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9531999999999999e-05 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483382+0000", 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6484999999999999e-05 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91015 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0569149070000003, 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059570633999999997, 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91015 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424047+0000", 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424050+0000", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5640000000000001e-06 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424053+0000", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483410+0000", 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059356475999999998 2024-09-17T22:58:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483489+0000", 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9263000000000003e-05 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483616+0000", 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012700299999999999 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91016 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 231424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.056902805, 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059795158000000001, 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91016 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424058+0000", 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0699999999999997e-07 2024-09-17T22:58:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424060+0000", 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8360000000000001e-06 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424068+0000", 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424072+0000", 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8249999999999998e-06 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483668+0000", 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059595608000000001 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483751+0000", 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3778999999999999e-05 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483852+0000", 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101059 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91017 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0568694440000002, 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059982901999999998, 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91017 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424096+0000", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0710000000000002e-06 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424103+0000", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424108+0000", 2024-09-17T22:58:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483886+0000", 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059777978000000002 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483971+0000", 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5172000000000004e-05 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484074+0000", 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102697 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91018 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0568441389999998, 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060220192999999998, 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91018 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424124+0000", 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0209999999999999e-06 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424135+0000", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0547000000000001e-05 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424143+0000", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484133+0000", 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059990452 2024-09-17T22:58:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484226+0000", 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2367999999999994e-05 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484336+0000", 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011052099999999999 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91019 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0568094349999999, 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060388084000000002, 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91019 2024-09-17T22:58:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424153+0000", 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2819999999999999e-06 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424162+0000", 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424168+0000", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484367+0000", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060199257999999999 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484446+0000", 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9029000000000005e-05 2024-09-17T22:58:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484539+0000", 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2687000000000002e-05 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91020 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.056785724, 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060513375000000001, 2024-09-17T22:58:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91020 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424175+0000", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424177+0000", 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.978e-06 2024-09-17T22:58:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424183+0000", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1109999999999996e-06 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424190+0000", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2300000000000002e-06 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484556+0000", 2024-09-17T22:58:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060365758999999998 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484614+0000", 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8233999999999998e-05 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484688+0000", 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3668e-05 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91021 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0567610780000001, 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060706860000000001, 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.547 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91021 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424201+0000", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.939e-06 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424207+0000", 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:58:22.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424217+0000", 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2e-06 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484719+0000", 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060502242999999997 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484795+0000", 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6462000000000001e-05 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484906+0000", 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011060700000000001 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91022 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:22.551 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0567368249999998, 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060885440999999998, 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91022 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424224+0000", 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424226+0000", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424232+0000", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1349999999999997e-06 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424240+0000", 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4320000000000002e-06 2024-09-17T22:58:22.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484935+0000", 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060695168000000001 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485019+0000", 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4040000000000002e-05 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485109+0000", 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9936000000000001e-05 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0567134280000001, 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:22.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0566926519999997, 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.561 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:22.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0566738620000002, 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:22.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0566286509999996, 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:22.574 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.0565948230000002, 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:22.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:22.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:22.579 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:22.580 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:22.581 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:22.578+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:22.681 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:22.679+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:22.725 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:22.725 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:22.725 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:22.725 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:22.733 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:22.782 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:22.779+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:22.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:22.877 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:22.877 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:22.878 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:22.878 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:22.883 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:22.881+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:22.886 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:22.931 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:22.932 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:22.948 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:22.949 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:22.949 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:22.965 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:22.965 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:22.983 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:22.981+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:22.984 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:22.984 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:23.003 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:23.003 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:23.021 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:23.021 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:23.036 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.036 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.038 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:23.042 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:23.042 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:23.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:23.046 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone37 from 0xc466925 (overlap 0x5ef5bb) to 0x25fbe4 2024-09-17T22:58:23.090 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:23.088+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:23.172 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.172 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.182 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:23.183 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T22:58:23.185 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:23.182+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:23.186 INFO:teuthology.orchestra.run.smithi159.stdout:4998 write 0xa0dee77 thru 0xa0e9a1a (0xaba4 bytes) 2024-09-17T22:58:23.194 INFO:teuthology.orchestra.run.smithi159.stdout:5001 writesame 0xa2e4ab3 thru 0xa2ed6b4 data_size 0x1(0x8c02 bytes) 2024-09-17T22:58:23.206 INFO:teuthology.orchestra.run.smithi159.stdout:5002 read 0x80bf57e thru 0x80ced1a (0xf79d bytes) 2024-09-17T22:58:23.207 INFO:teuthology.orchestra.run.smithi159.stdout:5003 write 0x3a1b8c8 thru 0x3a1c1d1 (0x90a bytes) 2024-09-17T22:58:23.210 INFO:teuthology.orchestra.run.smithi159.stdout:5004 punch from 0x748f83f to 0x7499936, (0xa0f7 bytes) 2024-09-17T22:58:23.210 INFO:teuthology.orchestra.run.smithi159.stdout:5005 writesame 0x322c2f thru 0x32f111 data_size 0x35(0xc4e3 bytes) 2024-09-17T22:58:23.231 INFO:teuthology.orchestra.run.smithi159.stdout:5007 trunc from 0xc466925 to 0xa474ad2 2024-09-17T22:58:23.254 INFO:teuthology.orchestra.run.smithi159.stdout:5008 trunc from 0xa474ad2 to 0x278929d 2024-09-17T22:58:23.274 INFO:teuthology.orchestra.run.smithi159.stdout:5011 read 0x13ce9fd thru 0x13d643a (0x7a3e bytes) 2024-09-17T22:58:23.274 INFO:teuthology.orchestra.run.smithi159.stdout:5012 read 0x22f9551 thru 0x23088cd (0xf37d bytes) 2024-09-17T22:58:23.274 INFO:teuthology.orchestra.run.smithi159.stdout:5013 read 0xd2e61e thru 0xd385ed (0x9fd0 bytes) 2024-09-17T22:58:23.274 INFO:teuthology.orchestra.run.smithi159.stdout:5014 write 0x2381355 thru 0x238f9a4 (0xe650 bytes) 2024-09-17T22:58:23.278 INFO:teuthology.orchestra.run.smithi159.stdout:5015 read 0x408841 thru 0x40d4eb (0x4cab bytes) 2024-09-17T22:58:23.279 INFO:teuthology.orchestra.run.smithi159.stdout:5016 read 0x772684 thru 0x7797ef (0x716c bytes) 2024-09-17T22:58:23.279 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.279 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.279 INFO:teuthology.orchestra.run.smithi159.stdout:5020 read 0x3068c thru 0x32de7 (0x275c bytes) 2024-09-17T22:58:23.285 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:23.283+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:23.291 INFO:teuthology.orchestra.run.smithi159.stdout:5022 write 0x81387e0 thru 0x8141336 (0x8b57 bytes) 2024-09-17T22:58:23.295 INFO:teuthology.orchestra.run.smithi159.stdout:5024 read 0x7c3d7bf thru 0x7c470c6 (0x9908 bytes) 2024-09-17T22:58:23.296 INFO:teuthology.orchestra.run.smithi159.stdout:5025 write 0x188e16b thru 0x189c39b (0xe231 bytes) 2024-09-17T22:58:23.303 INFO:teuthology.orchestra.run.smithi159.stdout:5026 write 0x1a4f0c1 thru 0x1a589fa (0x993a bytes) 2024-09-17T22:58:23.317 INFO:teuthology.orchestra.run.smithi159.stdout:5027 write 0x9797ce0 thru 0x97a78af (0xfbd0 bytes) 2024-09-17T22:58:23.323 INFO:teuthology.orchestra.run.smithi159.stdout:5028 writesame 0x51f0bd2 thru 0x51fd225 data_size 0x6(0xc654 bytes) 2024-09-17T22:58:23.330 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.330 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.330 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.330 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.336 INFO:teuthology.orchestra.run.smithi159.stdout:5034 write 0xa665cf4 thru 0xa668432 (0x273f bytes) 2024-09-17T22:58:23.339 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.339 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.341 INFO:teuthology.orchestra.run.smithi159.stdout:5035 trunc from 0xa668433 to 0x833bdb0 2024-09-17T22:58:23.352 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:23.353 INFO:teuthology.orchestra.run.smithi159.stdout:5037 read 0x1143ee1 thru 0x1153024 (0xf144 bytes) 2024-09-17T22:58:23.353 INFO:teuthology.orchestra.run.smithi159.stdout:5038 write 0x7a61423 thru 0x7a6f29b (0xde79 bytes) 2024-09-17T22:58:23.357 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.357 INFO:teuthology.orchestra.run.smithi159.stdout:5040 writesame 0x23e084d thru 0x23e5071 data_size 0xb(0x4825 bytes) 2024-09-17T22:58:23.360 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.364 INFO:teuthology.orchestra.run.smithi159.stdout:5043 trunc from 0x833bdb0 to 0x4760046 2024-09-17T22:58:23.374 INFO:teuthology.orchestra.run.smithi159.stdout:5045 punch from 0x44dde6a to 0x44df296, (0x142c bytes) 2024-09-17T22:58:23.374 INFO:teuthology.orchestra.run.smithi159.stdout:5046 read 0x34efe0b thru 0x34fb169 (0xb35f bytes) 2024-09-17T22:58:23.386 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:23.383+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:23.387 INFO:teuthology.orchestra.run.smithi159.stdout:5047 write 0xbc0fb01 thru 0xbc18daf (0x92af bytes) 2024-09-17T22:58:23.392 INFO:teuthology.orchestra.run.smithi159.stdout:5048 read 0x4ae9372 thru 0x4af7eb0 (0xeb3f bytes) 2024-09-17T22:58:23.392 INFO:teuthology.orchestra.run.smithi159.stdout:5049 write 0x7f049a1 thru 0x7f085f3 (0x3c53 bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5051 read 0x66628fa thru 0x66678f7 (0x4ffe bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5052 punch from 0xa042e6 to 0xa0d39e, (0x90b8 bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5053 read 0x417e329 thru 0x418ac02 (0xc8da bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5055 read 0x9a0ea7f thru 0x9a1d10e (0xe690 bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5058 read 0x9adc5f3 thru 0x9ade9eb (0x23f9 bytes) 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.396 INFO:teuthology.orchestra.run.smithi159.stdout:5065 punch from 0xb90d600 to 0xb90e8fc, (0x12fc bytes) 2024-09-17T22:58:23.406 INFO:teuthology.orchestra.run.smithi159.stdout:5067 trunc from 0xbc18db0 to 0x37aea2a 2024-09-17T22:58:23.416 INFO:teuthology.orchestra.run.smithi159.stdout:5069 read 0x3bd8ee thru 0x3c8648 (0xad5b bytes) 2024-09-17T22:58:23.416 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.430 INFO:teuthology.orchestra.run.smithi159.stdout:5072 writesame 0x9afa089 thru 0x9b067eb data_size 0x1(0xc763 bytes) 2024-09-17T22:58:23.436 INFO:teuthology.orchestra.run.smithi159.stdout:5073 writesame 0xa3f3696 thru 0xa3f8dd0 data_size 0x1(0x573b bytes) 2024-09-17T22:58:23.436 INFO:teuthology.orchestra.run.smithi159.stdout:5074 writesame 0x176daaf thru 0x177414b data_size 0x1(0x669d bytes) 2024-09-17T22:58:23.447 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.453 INFO:teuthology.orchestra.run.smithi159.stdout:5076 trunc from 0xa3f8dd1 to 0x4f57ecd 2024-09-17T22:58:23.459 INFO:teuthology.orchestra.run.smithi159.stdout:5078 punch from 0x9e66a4 to 0x9f42bc, (0xdc18 bytes) 2024-09-17T22:58:23.459 INFO:teuthology.orchestra.run.smithi159.stdout:5079 writesame 0xc6fa77 thru 0xc70f3e data_size 0x38(0x14c8 bytes) 2024-09-17T22:58:23.469 INFO:teuthology.orchestra.run.smithi159.stdout:5081 writesame 0x61f3e8b thru 0x61f761a data_size 0x1c(0x3790 bytes) 2024-09-17T22:58:23.485 INFO:teuthology.orchestra.run.smithi159.stdout:5082 writesame 0xea130ab thru 0xea1cd3c data_size 0x31(0x9c92 bytes) 2024-09-17T22:58:23.485 INFO:teuthology.orchestra.run.smithi159.stdout:5084 writesame 0xde742fa thru 0xde76c3a data_size 0x1(0x2941 bytes) 2024-09-17T22:58:23.486 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:23.484+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:23.492 INFO:teuthology.orchestra.run.smithi159.stdout:5085 read 0x858e582 thru 0x85965ee (0x806d bytes) 2024-09-17T22:58:23.492 INFO:teuthology.orchestra.run.smithi159.stdout:5086 write 0x64e9c94 thru 0x64ebb72 (0x1edf bytes) 2024-09-17T22:58:23.495 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:23.495 INFO:teuthology.orchestra.run.smithi159.stdout:5088 read 0x2bdf580 thru 0x2beb07c (0xbafd bytes) 2024-09-17T22:58:23.495 INFO:teuthology.orchestra.run.smithi159.stdout:5089 write 0xdaddb5c thru 0xdae7d99 (0xa23e bytes) 2024-09-17T22:58:23.499 INFO:teuthology.orchestra.run.smithi159.stdout:5090 punch from 0xd6b461a to 0xd6ba4d9, (0x5ebf bytes) 2024-09-17T22:58:23.500 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.501 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.502 INFO:teuthology.orchestra.run.smithi159.stdout:5091 trunc from 0xea1cd3d to 0xcd31227 2024-09-17T22:58:23.510 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:23.518 INFO:teuthology.orchestra.run.smithi159.stdout:5093 trunc from 0xcd31227 to 0x7fba92f 2024-09-17T22:58:23.523 INFO:teuthology.orchestra.run.smithi159.stdout:5094 clone 39 order 19 su 32768 sc 7 2024-09-17T22:58:23.587 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:23.585+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:23.656 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.657 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.665 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:23.709 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:23.707+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:23.767 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.767 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.776 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:23.813 INFO:teuthology.orchestra.run.smithi159.stderr:Exporting 2.3 info 2.3( v 19'1 (0'0,19'1] local-lis/les=59/60 n=1 ec=18/18 lis/c=59/59 les/c/f=60/60/0 sis=59) 2024-09-17T22:58:23.814 INFO:teuthology.orchestra.run.smithi159.stderr:Read #2:c4c92e5a:::rbd_trash:head# 2024-09-17T22:58:23.814 INFO:teuthology.orchestra.run.smithi159.stderr:Export successful 2024-09-17T22:58:23.817 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:23.815+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:23.890 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:23.888+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:23.921 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:23.921 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:23.930 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:23.981 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:24.025 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:24.023+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:24.045 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.045 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.055 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:24.060 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone38 from 0x7fba92f (overlap 0x278929d) to 0x23950c0 2024-09-17T22:58:24.091 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:24.089+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:24.181 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.181 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.189 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T22:58:24.190 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:24.191 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:24.189+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:24.255 INFO:teuthology.orchestra.run.smithi159.stdout:5095 read 0x17ffb47 thru 0x1801566 (0x1a20 bytes) 2024-09-17T22:58:24.256 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.256 INFO:teuthology.orchestra.run.smithi159.stdout:5097 write 0x10a5474 thru 0x10b2647 (0xd1d4 bytes) 2024-09-17T22:58:24.266 INFO:teuthology.orchestra.run.smithi159.stdout:5098 read 0x1881445 thru 0x1889e09 (0x89c5 bytes) 2024-09-17T22:58:24.266 INFO:teuthology.orchestra.run.smithi159.stdout:5099 read 0xb467fd thru 0xb53d0f (0xd513 bytes) 2024-09-17T22:58:24.266 INFO:teuthology.orchestra.run.smithi159.stdout:5100 read 0x748145b thru 0x7481d1b (0x8c1 bytes) 2024-09-17T22:58:24.266 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.266 INFO:teuthology.orchestra.run.smithi159.stdout:5103 read 0x7c76b53 thru 0x7c79db8 (0x3266 bytes) 2024-09-17T22:58:24.271 INFO:teuthology.orchestra.run.smithi159.stdout:5104 trunc from 0x7fba92f to 0x2f7163b 2024-09-17T22:58:24.286 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.286 INFO:teuthology.orchestra.run.smithi159.stdout:5106 read 0xdde344 thru 0xde514e (0x6e0b bytes) 2024-09-17T22:58:24.293 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:24.290+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:24.296 INFO:teuthology.orchestra.run.smithi159.stdout:5107 write 0x67cf8fe thru 0x67dc9bb (0xd0be bytes) 2024-09-17T22:58:24.300 INFO:teuthology.orchestra.run.smithi159.stdout:5108 read 0xcd7b49 thru 0xcd8eb9 (0x1371 bytes) 2024-09-17T22:58:24.300 INFO:teuthology.orchestra.run.smithi159.stdout:5109 punch from 0x3d303b8 to 0x3d31b3f, (0x1787 bytes) 2024-09-17T22:58:24.300 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.301 INFO:teuthology.orchestra.run.smithi159.stdout:5111 trunc from 0x67dc9bc to 0x7364e1d 2024-09-17T22:58:24.307 INFO:teuthology.orchestra.run.smithi159.stdout:5112 trunc from 0x7364e1d to 0x84b74e5 2024-09-17T22:58:24.313 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.319 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.319 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.323 INFO:teuthology.orchestra.run.smithi159.stdout:5114 write 0xc441203 thru 0xc44547f (0x427d bytes) 2024-09-17T22:58:24.330 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:24.330 INFO:teuthology.orchestra.run.smithi159.stdout:5115 trunc from 0xc445480 to 0x87e316c 2024-09-17T22:58:24.335 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 2.3 2024-09-17T22:58:24.340 INFO:teuthology.orchestra.run.smithi159.stdout:5116 read 0x465ad87 thru 0x465bd4d (0xfc7 bytes) 2024-09-17T22:58:24.340 INFO:teuthology.orchestra.run.smithi159.stdout:5118 punch from 0x3befbe2 to 0x3bf96ca, (0x9ae8 bytes) 2024-09-17T22:58:24.340 INFO:teuthology.orchestra.run.smithi159.stdout:5119 read 0x2a5bdb1 thru 0x2a603dc (0x462c bytes) 2024-09-17T22:58:24.352 INFO:teuthology.orchestra.run.smithi159.stdout:5120 write 0xd8299c6 thru 0xd833219 (0x9854 bytes) 2024-09-17T22:58:24.356 INFO:teuthology.orchestra.run.smithi159.stdout:5123 punch from 0x7002729 to 0x700886e, (0x6145 bytes) 2024-09-17T22:58:24.359 INFO:teuthology.orchestra.run.smithi159.stdout:5124 trunc from 0xd83321a to 0xb7b1507 2024-09-17T22:58:24.369 INFO:teuthology.orchestra.run.smithi159.stdout:5126 read 0x2bf5566 thru 0x2bfd128 (0x7bc3 bytes) 2024-09-17T22:58:24.369 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.369 INFO:teuthology.orchestra.run.smithi159.stdout:5129 write 0x8f2d4b1 thru 0x8f3aad3 (0xd623 bytes) 2024-09-17T22:58:24.378 INFO:teuthology.orchestra.run.smithi159.stdout:5130 read 0x368c49a thru 0x368df2a (0x1a91 bytes) 2024-09-17T22:58:24.388 INFO:teuthology.orchestra.run.smithi159.stdout:5131 trunc from 0xb7b1507 to 0x4168699 2024-09-17T22:58:24.393 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:24.391+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:24.398 INFO:teuthology.orchestra.run.smithi159.stdout:5132 write 0x39b7fa4 thru 0x39c068b (0x86e8 bytes) 2024-09-17T22:58:24.402 INFO:teuthology.orchestra.run.smithi159.stdout:5134 punch from 0xb05699 to 0xb0d519, (0x7e80 bytes) 2024-09-17T22:58:24.418 INFO:teuthology.orchestra.run.smithi159.stdout:5136 writesame 0xc1117bd thru 0xc113a98 data_size 0x2e(0x22dc bytes) 2024-09-17T22:58:24.418 INFO:teuthology.orchestra.run.smithi159.stdout:5137 read 0x8203bb9 thru 0x820a5b8 (0x6a00 bytes) 2024-09-17T22:58:24.418 INFO:teuthology.orchestra.run.smithi159.stdout:5138 punch from 0x73ea9a to 0x742aa0, (0x4006 bytes) 2024-09-17T22:58:24.418 INFO:teuthology.orchestra.run.smithi159.stdout:5139 punch from 0x878c5d5 to 0x878f3d6, (0x2e01 bytes) 2024-09-17T22:58:24.418 INFO:teuthology.orchestra.run.smithi159.stdout:5140 writesame 0xaa12738 thru 0xaa1dd23 data_size 0x6(0xb5ec bytes) 2024-09-17T22:58:24.423 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.423 INFO:teuthology.orchestra.run.smithi159.stdout:5142 read 0x9c36d9f thru 0x9c4493d (0xdb9f bytes) 2024-09-17T22:58:24.423 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.423 INFO:teuthology.orchestra.run.smithi159.stdout:5144 read 0x7a63b0 thru 0x7aeaa7 (0x86f8 bytes) 2024-09-17T22:58:24.423 INFO:teuthology.orchestra.run.smithi159.stdout:5146 read 0x59e427 thru 0x5a7677 (0x9251 bytes) 2024-09-17T22:58:24.424 INFO:teuthology.orchestra.run.smithi159.stdout:5147 write 0x3091d68 thru 0x309477f (0x2a18 bytes) 2024-09-17T22:58:24.425 INFO:teuthology.orchestra.run.smithi159.stdout:5148 read 0x6321379 thru 0x632f801 (0xe489 bytes) 2024-09-17T22:58:24.425 INFO:teuthology.orchestra.run.smithi159.stdout:5149 write 0x8dc6464 thru 0x8dc9df0 (0x398d bytes) 2024-09-17T22:58:24.434 INFO:teuthology.orchestra.run.smithi159.stdout:5150 write 0xc979a86 thru 0xc9813b3 (0x792e bytes) 2024-09-17T22:58:24.439 INFO:teuthology.orchestra.run.smithi159.stdout:5151 write 0x5efe859 thru 0x5f02344 (0x3aec bytes) 2024-09-17T22:58:24.444 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.444 INFO:teuthology.orchestra.run.smithi159.stdout:5154 writesame 0x6597f84 thru 0x659a2ce data_size 0xd(0x234b bytes) 2024-09-17T22:58:24.449 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.450 INFO:teuthology.orchestra.run.smithi159.stdout:5156 write 0x4fe1815 thru 0x4fe4a60 (0x324c bytes) 2024-09-17T22:58:24.454 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.454 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.454 INFO:teuthology.orchestra.run.smithi159.stdout:5157 write 0x8ba0ca2 thru 0x8ba8fd6 (0x8335 bytes) 2024-09-17T22:58:24.458 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.458 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.459 INFO:teuthology.orchestra.run.smithi159.stdout:5160 trunc from 0xc9813b4 to 0xb83a861 2024-09-17T22:58:24.465 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:24.471 INFO:teuthology.orchestra.run.smithi159.stdout:5161 punch from 0x9c6074f to 0x9c67ec5, (0x7776 bytes) 2024-09-17T22:58:24.472 INFO:teuthology.orchestra.run.smithi159.stdout:5164 read 0x5029b76 thru 0x50306ac (0x6b37 bytes) 2024-09-17T22:58:24.472 INFO:teuthology.orchestra.run.smithi159.stdout:5165 writesame 0xb488dee thru 0xb49328c data_size 0x25(0xa49f bytes) 2024-09-17T22:58:24.477 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.478 INFO:teuthology.orchestra.run.smithi159.stdout:5167 write 0x948a51a thru 0x948c345 (0x1e2c bytes) 2024-09-17T22:58:24.480 INFO:teuthology.orchestra.run.smithi159.stdout:5168 write 0x1c32f4f thru 0x1c3aec1 (0x7f73 bytes) 2024-09-17T22:58:24.484 INFO:teuthology.orchestra.run.smithi159.stdout:5169 writesame 0x9228fb thru 0x929d60 data_size 0x2f(0x7466 bytes) 2024-09-17T22:58:24.488 INFO:teuthology.orchestra.run.smithi159.stdout:5170 write 0x90110da thru 0x901d22f (0xc156 bytes) 2024-09-17T22:58:24.494 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:24.492+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:24.495 INFO:teuthology.orchestra.run.smithi159.stdout:5171 write 0x1e40cff thru 0x1e50ba4 (0xfea6 bytes) 2024-09-17T22:58:24.508 INFO:teuthology.orchestra.run.smithi159.stdout:5172 write 0xb9ccf4d thru 0xb9da407 (0xd4bb bytes) 2024-09-17T22:58:24.517 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.517 INFO:teuthology.orchestra.run.smithi159.stdout:5174 write 0x224584f thru 0x22461ac (0x95e bytes) 2024-09-17T22:58:24.521 INFO:teuthology.orchestra.run.smithi159.stdout:5175 flatten 2024-09-17T22:58:24.595 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:24.593+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:24.620 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.620 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.621 INFO:teuthology.orchestra.run.smithi159.stdout:5176 punch from 0x81c0029 to 0x81c544e, (0x5425 bytes) 2024-09-17T22:58:24.621 INFO:teuthology.orchestra.run.smithi159.stdout:5177 writesame 0xaad9d09 thru 0xaadc27e data_size 0x23(0x2576 bytes) 2024-09-17T22:58:24.626 INFO:teuthology.orchestra.run.smithi159.stdout:5178 punch from 0x12482c7 to 0x124de07, (0x5b40 bytes) 2024-09-17T22:58:24.626 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:24.634 INFO:teuthology.orchestra.run.smithi159.stdout:5181 writesame 0xd0430e3 thru 0xd04e79c data_size 0x13(0xb6ba bytes) 2024-09-17T22:58:24.634 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.634 INFO:teuthology.orchestra.run.smithi159.stdout:5183 write 0xb967459 thru 0xb970988 (0x9530 bytes) 2024-09-17T22:58:24.639 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.639 INFO:teuthology.orchestra.run.smithi159.stdout:5185 write 0x642e449 thru 0x6432069 (0x3c21 bytes) 2024-09-17T22:58:24.643 INFO:teuthology.orchestra.run.smithi159.stdout:5186 read 0x27031e4 thru 0x2704303 (0x1120 bytes) 2024-09-17T22:58:24.643 INFO:teuthology.orchestra.run.smithi159.stdout:5188 trunc from 0xd04e79d to 0xcd7f60e 2024-09-17T22:58:24.658 INFO:teuthology.orchestra.run.smithi159.stdout:5189 trunc from 0xcd7f60e to 0x5026e09 2024-09-17T22:58:24.674 INFO:teuthology.orchestra.run.smithi159.stdout:5190 punch from 0x3dfbf8f to 0x3dfcf9e, (0x100f bytes) 2024-09-17T22:58:24.674 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.674 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.674 INFO:teuthology.orchestra.run.smithi159.stdout:5196 read 0x3cebbca thru 0x3cf70f5 (0xb52c bytes) 2024-09-17T22:58:24.674 INFO:teuthology.orchestra.run.smithi159.stdout:5197 read 0x19d1228 thru 0x19db654 (0xa42d bytes) 2024-09-17T22:58:24.675 INFO:teuthology.orchestra.run.smithi159.stdout:5198 read 0x4efdbf9 thru 0x4f00793 (0x2b9b bytes) 2024-09-17T22:58:24.681 INFO:teuthology.orchestra.run.smithi159.stdout:5200 writesame 0x682dd49 thru 0x683c578 data_size 0x28(0xe830 bytes) 2024-09-17T22:58:24.682 INFO:teuthology.orchestra.run.smithi159.stdout:5202 read 0x502c298 thru 0x503150a (0x5273 bytes) 2024-09-17T22:58:24.686 INFO:teuthology.orchestra.run.smithi159.stdout:5203 write 0x68bd904 thru 0x68c2171 (0x486e bytes) 2024-09-17T22:58:24.690 INFO:teuthology.orchestra.run.smithi159.stdout:5204 punch from 0x4b6185e to 0x4b64a1d, (0x31bf bytes) 2024-09-17T22:58:24.690 INFO:teuthology.orchestra.run.smithi159.stdout:5205 writesame 0x1052211 thru 0x105c79d data_size 0x33(0xa58d bytes) 2024-09-17T22:58:24.696 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:24.694+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:24.700 INFO:teuthology.orchestra.run.smithi159.stdout:5206 punch from 0x7f2e62 to 0x7f5504, (0x26a2 bytes) 2024-09-17T22:58:24.700 INFO:teuthology.orchestra.run.smithi159.stdout:5207 write 0x6032f9 thru 0x610d37 (0xda3f bytes) 2024-09-17T22:58:24.705 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.720 INFO:teuthology.orchestra.run.smithi159.stdout:5209 write 0xe7092b0 thru 0xe70f07b (0x5dcc bytes) 2024-09-17T22:58:24.723 INFO:teuthology.orchestra.run.smithi159.stdout:5210 read 0x547648b thru 0x54817f2 (0xb368 bytes) 2024-09-17T22:58:24.724 INFO:teuthology.orchestra.run.smithi159.stdout:5211 punch from 0x18891cf to 0x18927c3, (0x95f4 bytes) 2024-09-17T22:58:24.724 INFO:teuthology.orchestra.run.smithi159.stdout:5213 write 0xd5468db thru 0xd54e278 (0x799e bytes) 2024-09-17T22:58:24.727 INFO:teuthology.orchestra.run.smithi159.stdout:5214 punch from 0xd4313f9 to 0xd440ed4, (0xfadb bytes) 2024-09-17T22:58:24.727 INFO:teuthology.orchestra.run.smithi159.stdout:5215 write 0x8a6e013 thru 0x8a72b33 (0x4b21 bytes) 2024-09-17T22:58:24.730 INFO:teuthology.orchestra.run.smithi159.stdout:5218 write 0x4bbbc6b thru 0x4bc0b20 (0x4eb6 bytes) 2024-09-17T22:58:24.733 INFO:teuthology.orchestra.run.smithi159.stdout:5220 write 0x79e5253 thru 0x79e60a7 (0xe55 bytes) 2024-09-17T22:58:24.736 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.736 INFO:teuthology.orchestra.run.smithi159.stdout:5224 writesame 0x5297925 thru 0x52a2388 data_size 0x1e(0xaa64 bytes) 2024-09-17T22:58:24.740 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.740 INFO:teuthology.orchestra.run.smithi159.stdout:5229 write 0x7875d90 thru 0x787e570 (0x87e1 bytes) 2024-09-17T22:58:24.744 INFO:teuthology.orchestra.run.smithi159.stdout:5230 write 0x219bafc thru 0x21a3407 (0x790c bytes) 2024-09-17T22:58:24.746 INFO:teuthology.orchestra.run.smithi159.stdout:5231 write 0x843c5d4 thru 0x843f661 (0x308e bytes) 2024-09-17T22:58:24.749 INFO:teuthology.orchestra.run.smithi159.stdout:5233 write 0xda1448a thru 0xda209df (0xc556 bytes) 2024-09-17T22:58:24.753 INFO:teuthology.orchestra.run.smithi159.stdout:5234 punch from 0xe057944 to 0xe062ca7, (0xb363 bytes) 2024-09-17T22:58:24.753 INFO:teuthology.orchestra.run.smithi159.stdout:5235 write 0x80a152b thru 0x80a596f (0x4445 bytes) 2024-09-17T22:58:24.755 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.755 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:24.773 INFO:teuthology.orchestra.run.smithi159.stdout:5236 trunc from 0xe70f07c to 0x124bb34 2024-09-17T22:58:24.797 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:24.795+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:24.812 INFO:teuthology.orchestra.run.smithi159.stdout:5237 writesame 0xac8e6fd thru 0xac981d2 data_size 0x1(0x9ad6 bytes) 2024-09-17T22:58:24.812 INFO:teuthology.orchestra.run.smithi159.stdout:5238 writesame 0x1bc3965 thru 0x1bc7a44 data_size 0x30(0x40e0 bytes) 2024-09-17T22:58:24.814 INFO:teuthology.orchestra.run.smithi159.stdout:5240 punch from 0xa7ce8aa to 0xa7d1852, (0x2fa8 bytes) 2024-09-17T22:58:24.816 INFO:teuthology.orchestra.run.smithi159.stdout:5241 trunc from 0xac981d3 to 0x9a8342d 2024-09-17T22:58:24.832 INFO:teuthology.orchestra.run.smithi159.stdout:5243 write 0xc903e4e thru 0xc907e1a (0x3fcd bytes) 2024-09-17T22:58:24.835 INFO:teuthology.orchestra.run.smithi159.stdout:5244 punch from 0x2ccbca3 to 0x2cd5c42, (0x9f9f bytes) 2024-09-17T22:58:24.835 INFO:teuthology.orchestra.run.smithi159.stdout:5246 write 0xbfe6709 thru 0xbfed8ca (0x71c2 bytes) 2024-09-17T22:58:24.841 INFO:teuthology.orchestra.run.smithi159.stdout:5248 punch from 0x6e206b3 to 0x6e2e00d, (0xd95a bytes) 2024-09-17T22:58:24.841 INFO:teuthology.orchestra.run.smithi159.stdout:5249 write 0x34611dd thru 0x3465caa (0x4ace bytes) 2024-09-17T22:58:24.853 INFO:teuthology.orchestra.run.smithi159.stdout:5250 write 0xe503f3f thru 0xe5068a8 (0x296a bytes) 2024-09-17T22:58:24.856 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.856 INFO:teuthology.orchestra.run.smithi159.stdout:5252 read 0x46451fc thru 0x464cf55 (0x7d5a bytes) 2024-09-17T22:58:24.856 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.856 INFO:teuthology.orchestra.run.smithi159.stdout:5254 read 0x6a4575 thru 0x6abab5 (0x7541 bytes) 2024-09-17T22:58:24.857 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.857 INFO:teuthology.orchestra.run.smithi159.stdout:5256 write 0xba13a3a thru 0xba21590 (0xdb57 bytes) 2024-09-17T22:58:24.874 INFO:teuthology.orchestra.run.smithi159.stdout:5257 trunc from 0xe5068a9 to 0x322784a 2024-09-17T22:58:24.895 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:24.895 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:24.898 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:24.896+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:24.902 INFO:teuthology.orchestra.run.smithi159.stdout:5258 write 0xccea96a thru 0xccf818e (0xd825 bytes) 2024-09-17T22:58:24.907 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:24.908 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.908 INFO:teuthology.orchestra.run.smithi159.stdout:5261 read 0x93dba4b thru 0x93e233c (0x68f2 bytes) 2024-09-17T22:58:24.908 INFO:teuthology.orchestra.run.smithi159.stdout:5264 write 0x3032b78 thru 0x303bffb (0x9484 bytes) 2024-09-17T22:58:24.910 INFO:teuthology.orchestra.run.smithi159.stdout:5265 write 0x13822eb thru 0x1384d1c (0x2a32 bytes) 2024-09-17T22:58:24.913 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.913 INFO:teuthology.orchestra.run.smithi159.stdout:5267 read 0xae084a7 thru 0xae0dad4 (0x562e bytes) 2024-09-17T22:58:24.914 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.914 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:24.914 INFO:teuthology.orchestra.run.smithi159.stdout:5270 read 0x5545da7 thru 0x554d2b8 (0x7512 bytes) 2024-09-17T22:58:24.914 INFO:teuthology.orchestra.run.smithi159.stdout:5271 clone 40 order 24 su 1048576 sc 4 2024-09-17T22:58:24.997 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:58:25.000 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:24.998+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:25.022 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:25.022 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:25.036 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:25.043 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:25.119 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:25.117+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:25.191 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:25.191 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:25.200 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:25.201 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:25.199+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:25.205 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:25.205 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:25.214 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T22:58:25.223 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:25.226 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:25.264 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:25.264 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:25.285 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:25.302 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:25.303 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:25.303 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:25.300+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:25.315 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:25.315 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:25.315 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:25.324 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:25.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:25.329 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:25.333 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:25.333 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:25.333 INFO:teuthology.orchestra.run.smithi159.stdout:5274 read 0x4674565 thru 0x4675898 (0x1334 bytes) 2024-09-17T22:58:25.334 INFO:teuthology.orchestra.run.smithi159.stdout:5275 writesame 0x3c17d3f thru 0x3c203ad data_size 0x1(0x866f bytes) 2024-09-17T22:58:25.348 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:25.349 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:25.352 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:25.352 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:25.366 INFO:teuthology.orchestra.run.smithi159.stdout:5276 write 0x4eb4e59 thru 0x4eb5bb2 (0xd5a bytes) 2024-09-17T22:58:25.370 INFO:teuthology.orchestra.run.smithi159.stdout:5278 read 0x57a8123 thru 0x57aa3c9 (0x22a7 bytes) 2024-09-17T22:58:25.371 INFO:teuthology.orchestra.run.smithi159.stdout:5279 write 0xed6f9d thru 0xedfcb5 (0x8d19 bytes) 2024-09-17T22:58:25.378 INFO:teuthology.orchestra.run.smithi159.stdout:5281 write 0x2a5f742 thru 0x2a6264c (0x2f0b bytes) 2024-09-17T22:58:25.379 INFO:teuthology.orchestra.run.smithi159.stdout:5282 write 0x978fabe thru 0x9793371 (0x38b4 bytes) 2024-09-17T22:58:25.391 INFO:teuthology.orchestra.run.smithi159.stdout:5284 writesame 0xdfd2813 thru 0xdfdf772 data_size 0x8(0xcf60 bytes) 2024-09-17T22:58:25.394 INFO:teuthology.orchestra.run.smithi159.stdout:5285 read 0x4114854 thru 0x41232b2 (0xea5f bytes) 2024-09-17T22:58:25.394 INFO:teuthology.orchestra.run.smithi159.stdout:5289 write 0xb644da0 thru 0xb64fdfd (0xb05e bytes) 2024-09-17T22:58:25.397 INFO:teuthology.orchestra.run.smithi159.stdout:5291 read 0xf50644 thru 0xf52694 (0x2051 bytes) 2024-09-17T22:58:25.397 INFO:teuthology.orchestra.run.smithi159.stdout:5292 clone 41 order 22 su 65536 sc 12 2024-09-17T22:58:25.403 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:25.401+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:25.430 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:25.431 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:25.439 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:25.504 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:25.502+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89486 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1955233150000009, 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059383219000000001, 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.584 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89486 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385240+0000", 2024-09-17T22:58:25.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385245+0000", 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.639e-06 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385248+0000", 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1319999999999998e-06 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444094+0000", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058845916999999998 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444207+0000", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113261 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444621+0000", 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041399299999999998 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.588 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89487 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1955088850000006, 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059624093000000003, 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.589 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89487 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.15e-07 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385254+0000", 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2389999999999999e-06 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385257+0000", 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385260+0000", 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.785e-06 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444667+0000", 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059407142000000003 2024-09-17T22:58:25.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444750+0000", 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2232999999999997e-05 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444877+0000", 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012703399999999999 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.593 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89488 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 257024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1954980800000001, 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059823605000000002, 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.594 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89488 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:25.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385264+0000", 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385265+0000", 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.206e-06 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385268+0000", 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385271+0000", 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7389999999999999e-06 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444898+0000", 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059626894999999999 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444967+0000", 2024-09-17T22:58:25.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9573999999999995e-05 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445087+0000", 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011978500000000001 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:25.600 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1954877479999997, 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:25.604 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91012 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1567936109999994, 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059095525000000003, 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91012 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423970+0000", 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423982+0000", 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423988+0000", 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482905+0000", 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058917737999999997 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482969+0000", 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4054999999999996e-05 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483063+0000", 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3968999999999994e-05 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91013 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1567634249999994, 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059240328000000002, 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91013 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:58:25.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424000+0000", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424012+0000", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1525000000000001e-05 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424018+0000", 2024-09-17T22:58:25.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483090+0000", 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059072066999999999 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483131+0000", 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1427000000000002e-05 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483238+0000", 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107161 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91014 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1567367419999997, 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059357291999999999, 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.614 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91014 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424027+0000", 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:58:25.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424035+0000", 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6389999999999992e-06 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424040+0000", 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9389999999999996e-06 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483266+0000", 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059226144000000001 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483326+0000", 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9531999999999999e-05 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483382+0000", 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6484999999999999e-05 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91015 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1567162750000008, 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059570633999999997, 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91015 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.619 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:25.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424047+0000", 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424050+0000", 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5640000000000001e-06 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424053+0000", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483410+0000", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059356475999999998 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483489+0000", 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9263000000000003e-05 2024-09-17T22:58:25.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483616+0000", 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012700299999999999 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91016 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 231424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1567041729999996, 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059795158000000001, 2024-09-17T22:58:25.623 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91016 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424058+0000", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0699999999999997e-07 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424060+0000", 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8360000000000001e-06 2024-09-17T22:58:25.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424068+0000", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424072+0000", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8249999999999998e-06 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483668+0000", 2024-09-17T22:58:25.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059595608000000001 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483751+0000", 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3778999999999999e-05 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483852+0000", 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101059 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91017 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1566708119999998, 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059982901999999998, 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91017 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.628 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.629 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424096+0000", 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0710000000000002e-06 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424103+0000", 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424108+0000", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483886+0000", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059777978000000002 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483971+0000", 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5172000000000004e-05 2024-09-17T22:58:25.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484074+0000", 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102697 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91018 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1566455070000004, 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060220192999999998, 2024-09-17T22:58:25.632 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91018 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424124+0000", 2024-09-17T22:58:25.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0209999999999999e-06 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424135+0000", 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0547000000000001e-05 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424143+0000", 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484133+0000", 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059990452 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484226+0000", 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2367999999999994e-05 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484336+0000", 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011052099999999999 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91019 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1566108029999995, 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060388084000000002, 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91019 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424153+0000", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2819999999999999e-06 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424162+0000", 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:58:25.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424168+0000", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484367+0000", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060199257999999999 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484446+0000", 2024-09-17T22:58:25.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9029000000000005e-05 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484539+0000", 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2687000000000002e-05 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91020 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1565870920000005, 2024-09-17T22:58:25.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060513375000000001, 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91020 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424175+0000", 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424177+0000", 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.978e-06 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424183+0000", 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1109999999999996e-06 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424190+0000", 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2300000000000002e-06 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484556+0000", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060365758999999998 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484614+0000", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8233999999999998e-05 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484688+0000", 2024-09-17T22:58:25.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3668e-05 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91021 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1565624460000006, 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060706860000000001, 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.646 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91021 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424201+0000", 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.939e-06 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424207+0000", 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424217+0000", 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2e-06 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484719+0000", 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060502242999999997 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484795+0000", 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6462000000000001e-05 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484906+0000", 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011060700000000001 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91022 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1565381929999994, 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060885440999999998, 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91022 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424224+0000", 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:25.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424226+0000", 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424232+0000", 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1349999999999997e-06 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424240+0000", 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4320000000000002e-06 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484935+0000", 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060695168000000001 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485019+0000", 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4040000000000002e-05 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.654 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485109+0000", 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9936000000000001e-05 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1565147959999997, 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.655 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:25.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1564940200000002, 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.660 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.664 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1564752299999999, 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:25.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1564300190000001, 2024-09-17T22:58:25.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:25.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:25.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.1563961910000007, 2024-09-17T22:58:25.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:25.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:25.678 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:25.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:25.679 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:25.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:25.681 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:25.678+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:25.781 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:25.779+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:25.824 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:25.824 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:25.825 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:25.825 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:25.834 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:25.882 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:25.879+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:25.937 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:25.938 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:25.938 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:25.938 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:25.938 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:25.946 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:25.982 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:25.980+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:26.007 INFO:teuthology.orchestra.run.smithi159.stdout: marking collection for removal 2024-09-17T22:58:26.008 INFO:teuthology.orchestra.run.smithi159.stdout:setting '_remove' omap key 2024-09-17T22:58:26.008 INFO:teuthology.orchestra.run.smithi159.stdout:finish_remove_pgs 2.3_head removing 2.3 2024-09-17T22:58:26.008 INFO:teuthology.orchestra.run.smithi159.stdout:Remove successful 2024-09-17T22:58:26.057 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone40 from 0xdfdf773 (overlap 0xccf818f) to 0x28fe6b6 2024-09-17T22:58:26.083 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:26.081+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844468804999998, 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844452799999999, 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844420030000002, 2024-09-17T22:58:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844392292000002, 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:26.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844369872999998, 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:26.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844347593000002, 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:26.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.844140187999997, 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.128 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.843593573, 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.842522377000002, 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:26.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.842510922000002, 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:26.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.842499945, 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:26.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:26.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:26.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.840425813000003, 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:26.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:26.154 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.840413499, 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.155 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.840400262000003, 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.159 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:26.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:26.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.840389432999999, 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:26.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:26.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.167 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.838430557999999, 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:26.171 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.838406200000001, 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.172 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.175 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.176 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.838376982, 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.177 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:26.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:26.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.837610413999997, 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:26.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 49.837597312, 2024-09-17T22:58:26.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:26.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:26.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.190 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.190 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:26.190 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T22:58:26.191 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:26.192 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:26.188+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:26.292 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:26.290+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:26.335 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:26.335 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:26.336 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:26.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:26.368 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:26.368 INFO:teuthology.orchestra.run.smithi159.stdout:5294 read 0x81c03cb thru 0x81cbbeb (0xb821 bytes) 2024-09-17T22:58:26.369 INFO:teuthology.orchestra.run.smithi159.stdout:5297 read 0xce98877 thru 0xcea56e6 (0xce70 bytes) 2024-09-17T22:58:26.384 INFO:teuthology.orchestra.run.smithi159.stdout:5298 trunc from 0xdfdf773 to 0x18e1ef3 2024-09-17T22:58:26.428 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:26.426+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:26.450 INFO:teuthology.orchestra.run.smithi159.stdout:5299 write 0x29409b thru 0x29f6b3 (0xb619 bytes) 2024-09-17T22:58:26.452 INFO:teuthology.orchestra.run.smithi159.stdout:5300 write 0x1688693 thru 0x168fe9d (0x780b bytes) 2024-09-17T22:58:26.453 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:26.464 INFO:teuthology.orchestra.run.smithi159.stdout:5303 writesame 0x6a39cf7 thru 0x6a3b043 data_size 0x1b(0x134d bytes) 2024-09-17T22:58:26.472 INFO:teuthology.orchestra.run.smithi159.stdout:5304 write 0x81ee48e thru 0x81f63ba (0x7f2d bytes) 2024-09-17T22:58:26.479 INFO:teuthology.orchestra.run.smithi159.stdout:5305 read 0x6a885e6 thru 0x6a92fea (0xaa05 bytes) 2024-09-17T22:58:26.479 INFO:teuthology.orchestra.run.smithi159.stdout:5307 write 0x5b2c3f1 thru 0x5b325c7 (0x61d7 bytes) 2024-09-17T22:58:26.493 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:26.491+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:26.496 INFO:teuthology.orchestra.run.smithi159.stdout:5308 write 0xd479111 thru 0xd480666 (0x7556 bytes) 2024-09-17T22:58:26.501 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:26.501 INFO:teuthology.orchestra.run.smithi159.stdout:5310 clone 42 order 18 su 65536 sc 2 2024-09-17T22:58:26.502 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:26.502 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:26.502 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:26.502 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:26.502 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:26.511 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:26.518 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.3.2 2024-09-17T22:58:26.603 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:26.601+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:26.656 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.792445071000003, 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:26.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:26.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:26.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7061 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565761731000002, 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304629000000001, 2024-09-17T22:58:26.663 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7061 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087285+0000", 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.665 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087296+0000", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0801e-05 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087306+0000", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9709999999999999e-06 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087311+0000", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1420000000000001e-06 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134397+0000", 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047085677999999999 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134482+0000", 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5462000000000003e-05 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134589+0000", 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107088 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.667 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7062 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4194304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565727195000001, 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047508084999999999, 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7062 2024-09-17T22:58:26.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087319+0000", 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087320+0000", 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087322+0000", 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087332+0000", 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0312000000000001e-05 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087338+0000", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2569999999999998e-06 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134642+0000", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047304023000000001 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134732+0000", 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0043999999999994e-05 2024-09-17T22:58:26.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134827+0000", 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5787999999999998e-05 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7063 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4390912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565672812000003, 2024-09-17T22:58:26.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047632724000000001, 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7063 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087374+0000", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087378+0000", 2024-09-17T22:58:26.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087384+0000", 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087390+0000", 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5060000000000003e-06 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134852+0000", 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047462566999999997 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.134920+0000", 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8275999999999995e-05 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135006+0000", 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6001000000000006e-05 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.676 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7064 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4587520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565647808000001, 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047775585000000002, 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.677 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7064 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087399+0000", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087401+0000", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087408+0000", 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2179999999999996e-06 2024-09-17T22:58:26.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087413+0000", 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4129999999999997e-06 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135044+0000", 2024-09-17T22:58:26.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047630643 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135113+0000", 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8974999999999996e-05 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135174+0000", 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1773000000000006e-05 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7065 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4784128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565627227, 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047973309999999998, 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7065 2024-09-17T22:58:26.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087419+0000", 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087420+0000", 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087422+0000", 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087429+0000", 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6410000000000002e-06 2024-09-17T22:58:26.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087435+0000", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2739999999999998e-06 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135221+0000", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047786640999999998 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135306+0000", 2024-09-17T22:58:26.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4556999999999997e-05 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135393+0000", 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6808e-05 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:26.686 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565605943999998, 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:26.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:26.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565587418, 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:26.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.695 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565569246000003, 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:26.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:26.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565551417999998, 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.565236284999997, 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:26.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7071 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.564514013999997, 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047735993999999997, 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7071 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088533+0000", 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4600000000000004e-07 2024-09-17T22:58:26.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088537+0000", 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.811e-06 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088547+0000", 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0509000000000001e-05 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088553+0000", 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1490000000000002e-06 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136163+0000", 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047609957000000001 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136208+0000", 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5000000000000003e-05 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136269+0000", 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0921999999999997e-05 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7072 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.564469418000002, 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047835152999999998, 2024-09-17T22:58:26.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7072 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088577+0000", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088578+0000", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6499999999999999e-07 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088581+0000", 2024-09-17T22:58:26.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3859999999999999e-06 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088589+0000", 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2700000000000004e-06 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088595+0000", 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0689999999999998e-06 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136294+0000", 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047698795000000002 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136340+0000", 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5420000000000002e-05 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136412+0000", 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2648e-05 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7073 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 6356992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.564442096000001, 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047957076000000001, 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7073 2024-09-17T22:58:26.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088604+0000", 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090232+0000", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001627912 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090242+0000", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0020000000000001e-05 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090253+0000", 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0169e-05 2024-09-17T22:58:26.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090259+0000", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7309999999999996e-06 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136446+0000", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046186506000000002 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136501+0000", 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5157000000000001e-05 2024-09-17T22:58:26.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136562+0000", 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0581000000000002e-05 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7081 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 7929856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.562623297999998, 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047470580999999998, 2024-09-17T22:58:26.723 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7081 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090423+0000", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090714+0000", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000290372 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090718+0000", 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:26.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090728+0000", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090735+0000", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.601e-06 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137713+0000", 2024-09-17T22:58:26.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.046978451999999997 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137793+0000", 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0531000000000005e-05 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.137894+0000", 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010033499999999999 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7083 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 8323072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.562280493000003, 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047384607000000002, 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7083 2024-09-17T22:58:26.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.090766+0000", 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092713+0000", 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019471110000000001 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092719+0000", 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092736+0000", 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7842999999999998e-05 2024-09-17T22:58:26.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.092745+0000", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.191e-06 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138041+0000", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045296627999999999 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138088+0000", 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6678999999999998e-05 2024-09-17T22:58:26.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.138151+0000", 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2798000000000004e-05 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7096 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 10813440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.560009360999999, 2024-09-17T22:58:26.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047326202999999997, 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7096 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093037+0000", 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093038+0000", 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.46e-07 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093039+0000", 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9740000000000001e-06 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093045+0000", 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7230000000000001e-06 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093050+0000", 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3980000000000001e-06 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140262+0000", 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047212386000000002 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140308+0000", 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6397000000000003e-05 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140363+0000", 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4979000000000003e-05 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7097 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11010048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.559977267000001, 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047417793999999999, 2024-09-17T22:58:26.737 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7097 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093069+0000", 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093070+0000", 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093077+0000", 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093084+0000", 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9779999999999999e-06 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093089+0000", 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:58:26.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140386+0000", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047297517999999997 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140431+0000", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5005e-05 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140487+0000", 2024-09-17T22:58:26.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5714000000000002e-05 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7098 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11206656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.559950729000001, 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047510587999999999, 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7098 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093096+0000", 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0800000000000005e-07 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093098+0000", 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093104+0000", 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6980000000000003e-06 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093109+0000", 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140509+0000", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047400104999999998 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140553+0000", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3707e-05 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140606+0000", 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3863000000000003e-05 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.746 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7099 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 11403264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.559931835999997, 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047622681, 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:26.747 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7099 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.093115+0000", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094714+0000", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015996599999999999 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094718+0000", 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:58:26.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094728+0000", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.094735+0000", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0040000000000002e-06 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140629+0000", 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.045893653 2024-09-17T22:58:26.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140673+0000", 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4211000000000001e-05 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.140737+0000", 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4177999999999994e-05 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:26.751 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:26.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:26.753 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:26.751+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:26.854 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:26.852+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:26.897 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:26.897 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:26.897 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:26.897 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:26.905 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:26.955 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:26.953+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:27.013 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:58:27.014 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:27.014 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:27.014 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:27.023 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:27.059 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:27.057+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:27.064 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone41 from 0xd480667 (overlap 0x18e1ef3) to 0x4f9fca 2024-09-17T22:58:27.134 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:58:27.134 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:27.143 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:27.144 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T22:58:27.187 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:27.185+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:27.200 INFO:teuthology.orchestra.run.smithi159.stdout:5311 punch from 0x63d67ad to 0x63dc6b3, (0x5f06 bytes) 2024-09-17T22:58:27.204 INFO:teuthology.orchestra.run.smithi159.stdout:5313 read 0x43c7a85 thru 0x43d6862 (0xedde bytes) 2024-09-17T22:58:27.204 INFO:teuthology.orchestra.run.smithi159.stdout:5314 write 0xd39ede5 thru 0xd3ac5b0 (0xd7cc bytes) 2024-09-17T22:58:27.210 INFO:teuthology.orchestra.run.smithi159.stdout:5315 read 0x7fa6f65 thru 0x7faf991 (0x8a2d bytes) 2024-09-17T22:58:27.210 INFO:teuthology.orchestra.run.smithi159.stdout:5316 write 0x3217677 thru 0x321ee73 (0x77fd bytes) 2024-09-17T22:58:27.214 INFO:teuthology.orchestra.run.smithi159.stdout:5317 writesame 0x78bd368 thru 0x78c5e2c data_size 0x31(0x8ac5 bytes) 2024-09-17T22:58:27.220 INFO:teuthology.orchestra.run.smithi159.stdout:5319 write 0x2ed84bd thru 0x2ee12f0 (0x8e34 bytes) 2024-09-17T22:58:27.230 INFO:teuthology.orchestra.run.smithi159.stdout:5321 trunc from 0xd480667 to 0xb7c7e55 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5323 writesame 0xd22d320 thru 0xd2325f0 data_size 0x25(0x52d1 bytes) 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5325 punch from 0x4cabc96 to 0x4cac072, (0x3dc bytes) 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5326 read 0x9cbe2c6 thru 0x9cc12b6 (0x2ff1 bytes) 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5328 punch from 0x703eec4 to 0x70407b8, (0x18f4 bytes) 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5329 read 0x501e734 thru 0x5020b1a (0x23e7 bytes) 2024-09-17T22:58:27.255 INFO:teuthology.orchestra.run.smithi159.stdout:5330 writesame 0x70e7d7e thru 0x70ef13b data_size 0x5(0x73be bytes) 2024-09-17T22:58:27.258 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:27.255+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:27.261 INFO:teuthology.orchestra.run.smithi159.stdout:5331 write 0xc6951f7 thru 0xc69dbd8 (0x89e2 bytes) 2024-09-17T22:58:27.272 INFO:teuthology.orchestra.run.smithi159.stdout:5332 trunc from 0xd2325f1 to 0x855e12f 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:121386 3.1 3:879919ef:::rbd_object_map.1188eaa36ced:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:27.277181+0000", 2024-09-17T22:58:27.279 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 7.8349000000000005e-05, 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8288999999999994e-05, 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 121386 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.280 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277181+0000", 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277181+0000", 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:27.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277181+0000", 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8400000000000003e-07 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277186+0000", 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277199+0000", 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2362e-05 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277206+0000", 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277248+0000", 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1613999999999998e-05 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:121387 3.1 3.f79899e1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:27.277228+0000", 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 3.0497e-05, 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0431000000000002e-05, 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:27.284 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 121387 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277228+0000", 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:27.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277228+0000", 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277229+0000", 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5199999999999996e-07 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277233+0000", 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6509999999999999e-06 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277238+0000", 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:27.277241+0000", 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.489e-06 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:58:27.287 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 2 2024-09-17T22:58:27.288 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:27.288 INFO:teuthology.orchestra.run.smithi159.stdout:5334 writesame 0x707dd13 thru 0x707e51d data_size 0x1(0x80b bytes) 2024-09-17T22:58:27.289 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:27.295 INFO:teuthology.orchestra.run.smithi159.stdout:5335 trunc from 0x855e12f to 0xc583728 2024-09-17T22:58:27.300 INFO:teuthology.orchestra.run.smithi159.stdout:5336 punch from 0x5f48600 to 0x5f4f9fc, (0x73fc bytes) 2024-09-17T22:58:27.309 INFO:teuthology.orchestra.run.smithi159.stdout:5337 trunc from 0xc583728 to 0x4666224 2024-09-17T22:58:27.324 INFO:teuthology.orchestra.run.smithi159.stdout:5339 read 0x3104c81 thru 0x310d001 (0x8381 bytes) 2024-09-17T22:58:27.324 INFO:teuthology.orchestra.run.smithi159.stdout:5340 read 0xb47a34 thru 0xb547f5 (0xcdc2 bytes) 2024-09-17T22:58:27.330 INFO:teuthology.orchestra.run.smithi159.stdout:5341 write 0x519d0d1 thru 0x519ff83 (0x2eb3 bytes) 2024-09-17T22:58:27.347 INFO:teuthology.orchestra.run.smithi159.stdout:5343 write 0xc2302ff thru 0xc232359 (0x205b bytes) 2024-09-17T22:58:27.350 INFO:teuthology.orchestra.run.smithi159.stdout:5345 write 0xb3a455b thru 0xb3b404f (0xfaf5 bytes) 2024-09-17T22:58:27.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:27.356 INFO:teuthology.orchestra.run.smithi159.stdout:5346 write 0x80f3db8 thru 0x80f4ea0 (0x10e9 bytes) 2024-09-17T22:58:27.358 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:27.356+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:27.368 INFO:teuthology.orchestra.run.smithi159.stdout:5347 trunc from 0xc23235a to 0x3134e0f 2024-09-17T22:58:27.403 INFO:teuthology.orchestra.run.smithi159.stdout:5348 writesame 0xb5e48ed thru 0xb5efd07 data_size 0x2f(0xb41b bytes) 2024-09-17T22:58:27.403 INFO:teuthology.orchestra.run.smithi159.stdout:5349 read 0x41ad8a8 thru 0x41bc180 (0xe8d9 bytes) 2024-09-17T22:58:27.411 INFO:teuthology.orchestra.run.smithi159.stdout:5350 write 0xd98aed6 thru 0xd991cd0 (0x6dfb bytes) 2024-09-17T22:58:27.415 INFO:teuthology.orchestra.run.smithi159.stdout:5351 write 0x52e07bf thru 0x52e9ddf (0x9621 bytes) 2024-09-17T22:58:27.418 INFO:teuthology.orchestra.run.smithi159.stdout:5353 flatten 2024-09-17T22:58:27.428 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:27.428 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:27.428 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:27.428 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:27.428 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:27.444 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:27.481 INFO:teuthology.orchestra.run.smithi159.stdout:5355 write 0x5ab13be thru 0x5ab2928 (0x156b bytes) 2024-09-17T22:58:27.482 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:27.479+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:27.487 INFO:teuthology.orchestra.run.smithi159.stdout:5357 trunc from 0xd991cd1 to 0xb21f205 2024-09-17T22:58:27.499 INFO:teuthology.orchestra.run.smithi159.stdout:5360 read 0x203c70e thru 0x2041dcb (0x56be bytes) 2024-09-17T22:58:27.499 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:27.499 INFO:teuthology.orchestra.run.smithi159.stdout:5363 write 0x73ad486 thru 0x73b493f (0x74ba bytes) 2024-09-17T22:58:27.503 INFO:teuthology.orchestra.run.smithi159.stdout:5364 clone 43 order 18 su 8192 sc 3 2024-09-17T22:58:27.513 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:27.514 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:27.530 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:27.531 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:27.531 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:27.547 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:27.547 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:27.561 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:27.559+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:27.564 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:27.564 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:27.581 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:27.581 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:27.589 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:27.589 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:27.597 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:27.597 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:27.598 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:27.614 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:27.614 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:27.617 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:27.617 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:27.662 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:27.659+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:27.718 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:27.718 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:27.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:27.763 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:27.761+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:27.828 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:27.828 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:27.836 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:27.880 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:27.878+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:27.940 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:27.941 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:27.949 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:27.991 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:27.988+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:28.006 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:58:28.066 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:28.064+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:28.105 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.105 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.115 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:28.167 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:28.165+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:28.204 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T22:58:28.206 INFO:teuthology.orchestra.run.smithi159.stdout:get_pg_num_history pg_num_history pg_num_history(e74 pg_nums {1={15=1},2={18=8},3={22=32}} deleted_pools ) 2024-09-17T22:58:28.206 INFO:teuthology.orchestra.run.smithi159.stdout:Importing pgid 2.3 2024-09-17T22:58:28.206 INFO:teuthology.orchestra.run.smithi159.stdout:Write #2:c4c92e5a:::rbd_trash:head# 2024-09-17T22:58:28.207 INFO:teuthology.orchestra.run.smithi159.stdout:snapset 0=[]:{} 2024-09-17T22:58:28.211 INFO:teuthology.orchestra.run.smithi159.stdout:write_pg epoch 60 info 2.3( v 19'1 (0'0,19'1] local-lis/les=59/60 n=1 ec=18/18 lis/c=59/59 les/c/f=60/60/0 sis=59) 2024-09-17T22:58:28.212 INFO:teuthology.orchestra.run.smithi159.stdout:Import successful 2024-09-17T22:58:28.216 INFO:teuthology.orchestra.run.smithi159.stdout:5365 write 0x65d7ea5 thru 0x65e5bd9 (0xdd35 bytes) 2024-09-17T22:58:28.224 INFO:teuthology.orchestra.run.smithi159.stdout:5368 read 0x642083a thru 0x642847f (0x7c46 bytes) 2024-09-17T22:58:28.233 INFO:teuthology.orchestra.run.smithi159.stdout:5369 trunc from 0xb21f205 to 0x393587e 2024-09-17T22:58:28.247 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:28.248 INFO:teuthology.orchestra.run.smithi159.stdout:5372 clone 44 order 19 su 262144 sc 7 2024-09-17T22:58:28.268 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:28.266+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:28.273 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.273 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.281 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:28.369 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:28.367+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:28.384 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.384 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.392 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:28.470 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:28.468+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:28.500 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.501 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.509 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:28.593 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:28.591+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:28.651 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.651 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:28.701 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:28.699+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:28.723 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /var/log/ceph/exp.2.3.2 2024-09-17T22:58:28.789 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:28.787+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:28.803 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.803 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.811 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:28.873 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:28.871+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:28.915 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:28.915 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:28.924 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:29.009 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:29.007+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:29.074 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:29.072+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:29.076 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.076 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.081 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone43 from 0x393587e (overlap 0x393587e) to 0x280bdf1 2024-09-17T22:58:29.086 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:29.176 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:29.174+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:29.190 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T22:58:29.234 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.234 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.245 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:29.277 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:29.275+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:29.378 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:29.376+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:29.391 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.391 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.397 INFO:teuthology.orchestra.run.smithi159.stdout:5373 write 0x377f7a0 thru 0x3786d08 (0x7569 bytes) 2024-09-17T22:58:29.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:29.405 INFO:teuthology.orchestra.run.smithi159.stdout:5375 punch from 0x252f18e to 0x253eb76, (0xf9e8 bytes) 2024-09-17T22:58:29.417 INFO:teuthology.orchestra.run.smithi159.stdout:5377 writesame 0x9451661 thru 0x946092f data_size 0x1(0xf2cf bytes) 2024-09-17T22:58:29.428 INFO:teuthology.orchestra.run.smithi159.stdout:5378 write 0xec92ec9 thru 0xec9374d (0x885 bytes) 2024-09-17T22:58:29.443 INFO:teuthology.orchestra.run.smithi159.stdout:5379 trunc from 0xec9374e to 0x4e248ee 2024-09-17T22:58:29.452 INFO:teuthology.orchestra.run.smithi159.stdout:5380 punch from 0x192adf3 to 0x19341b8, (0x93c5 bytes) 2024-09-17T22:58:29.470 INFO:teuthology.orchestra.run.smithi159.stdout:5381 writesame 0xe816ef9 thru 0xe823118 data_size 0x10(0xc220 bytes) 2024-09-17T22:58:29.470 INFO:teuthology.orchestra.run.smithi159.stdout:5382 writesame 0x78fc2b thru 0x79bd72 data_size 0x14(0xc148 bytes) 2024-09-17T22:58:29.476 INFO:teuthology.orchestra.run.smithi159.stdout:5383 write 0x841e5b thru 0x846576 (0x471c bytes) 2024-09-17T22:58:29.480 INFO:teuthology.orchestra.run.smithi159.stdout:5384 writesame 0xaa0974 thru 0xaaea93 data_size 0x10(0xe120 bytes) 2024-09-17T22:58:29.480 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:29.478+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:29.487 INFO:teuthology.orchestra.run.smithi159.stdout:5385 writesame 0x55deb33 thru 0x55ec2ef data_size 0x1(0xd7bd bytes) 2024-09-17T22:58:29.517 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:29.517 INFO:teuthology.orchestra.run.smithi159.stdout:5387 read 0x606e05d thru 0x607dcc0 (0xfc64 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5388 punch from 0x55c4acc to 0x55c5dc1, (0x12f5 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5389 read 0x8ee272d thru 0x8eefbe1 (0xd4b5 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5390 read 0xe51c72a thru 0xe52640c (0x9ce3 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5392 read 0x3aad199 thru 0x3aaf5b8 (0x2420 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5394 read 0xc84b7de thru 0xc84be75 (0x698 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5395 punch from 0x9966bbc to 0x996c105, (0x5549 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5399 read 0x4f4b24 thru 0x4ff6e8 (0xabc5 bytes) 2024-09-17T22:58:29.518 INFO:teuthology.orchestra.run.smithi159.stdout:5402 writesame 0x2b11d5e thru 0x2b21cc2 data_size 0x1(0xff65 bytes) 2024-09-17T22:58:29.547 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.547 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.552 INFO:teuthology.orchestra.run.smithi159.stdout:5403 clone 45 order 21 su 262144 sc 8 2024-09-17T22:58:29.556 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:29.582 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:29.579+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:29.618 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:29.659 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.659 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.667 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:29.711 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:29.708+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:29.738 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:29.739 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:29.755 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:29.756 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:29.756 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:29.771 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:29.771 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:29.772 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:29.772 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:29.780 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:29.787 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:29.788 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:29.804 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:29.804 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:29.820 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:29.821 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:29.821 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:29.818+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:29.837 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:29.837 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:29.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:29.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:29.884 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:29.881+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:29.925 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:29.925 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:29.925 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:29.925 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:29.925 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:29.934 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:29.985 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:29.982+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:30.023 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:58:30.083 INFO:teuthology.orchestra.run.smithi159.stdout:leaving image image_client.0-clone44 intact 2024-09-17T22:58:30.085 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:30.085 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:58:30.085 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:30.086 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:30.083+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:30.094 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:30.187 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:30.184+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:30.205 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.205 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:30.205 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:30.205 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:30.215 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:30.287 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:30.285+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:30.293 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-17T22:58:30.340 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.340 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:30.340 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:30.340 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:30.340 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:30.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:30.377 INFO:teuthology.orchestra.run.smithi159.stdout:5405 writesame 0x5658518 thru 0x56589fd data_size 0x16(0x4e6 bytes) 2024-09-17T22:58:30.383 INFO:teuthology.orchestra.run.smithi159.stdout:5406 write 0xd8a57dd thru 0xd8b430e (0xeb32 bytes) 2024-09-17T22:58:30.387 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:30.388 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:30.385+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:30.388 INFO:teuthology.orchestra.run.smithi159.stdout:5408 trunc from 0xe823119 to 0xd0edbc1 2024-09-17T22:58:30.399 INFO:teuthology.orchestra.run.smithi159.stdout:5410 read 0x7bddea2 thru 0x7be0546 (0x26a5 bytes) 2024-09-17T22:58:30.399 INFO:teuthology.orchestra.run.smithi159.stdout:5411 read 0x70d67f6 thru 0x70dc6d6 (0x5ee1 bytes) 2024-09-17T22:58:30.399 INFO:teuthology.orchestra.run.smithi159.stdout:5412 read 0x45c92bc thru 0x45c9710 (0x455 bytes) 2024-09-17T22:58:30.399 INFO:teuthology.orchestra.run.smithi159.stdout:5413 writesame 0x40a309d thru 0x40ae39a data_size 0x15(0xb2fe bytes) 2024-09-17T22:58:30.402 INFO:teuthology.orchestra.run.smithi159.stdout:5415 writesame 0x69ba858 thru 0x69c97f7 data_size 0x6c(0xefa0 bytes) 2024-09-17T22:58:30.406 INFO:teuthology.orchestra.run.smithi159.stdout:5416 read 0xca46f54 thru 0xca5271c (0xb7c9 bytes) 2024-09-17T22:58:30.409 INFO:teuthology.orchestra.run.smithi159.stdout:5417 trunc from 0xd0edbc1 to 0xab94bb2 2024-09-17T22:58:30.426 INFO:teuthology.orchestra.run.smithi159.stdout:5420 write 0x4c2c51b thru 0x4c2ef1a (0x2a00 bytes) 2024-09-17T22:58:30.428 INFO:teuthology.orchestra.run.smithi159.stdout:5421 read 0x4698e9f thru 0x46a47fd (0xb95f bytes) 2024-09-17T22:58:30.429 INFO:teuthology.orchestra.run.smithi159.stdout:5423 write 0x5c41749 thru 0x5c485d8 (0x6e90 bytes) 2024-09-17T22:58:30.430 INFO:teuthology.orchestra.run.smithi159.stdout:5424 write 0x6fd70cd thru 0x6fe1cbe (0xabf2 bytes) 2024-09-17T22:58:30.432 INFO:teuthology.orchestra.run.smithi159.stdout:5425 read 0x52e89c5 thru 0x52f02ce (0x790a bytes) 2024-09-17T22:58:30.432 INFO:teuthology.orchestra.run.smithi159.stdout:5426 write 0x62b7b6 thru 0x62ef6a (0x37b5 bytes) 2024-09-17T22:58:30.446 INFO:teuthology.orchestra.run.smithi159.stdout:5428 read 0x1d336d8 thru 0x1d34e4e (0x1777 bytes) 2024-09-17T22:58:30.454 INFO:teuthology.orchestra.run.smithi159.stdout:5429 writesame 0xc58deb9 thru 0xc59258a data_size 0x23(0x46d2 bytes) 2024-09-17T22:58:30.465 INFO:teuthology.orchestra.run.smithi159.stdout:5430 trunc from 0xc59258b to 0x3402368 2024-09-17T22:58:30.485 INFO:teuthology.orchestra.run.smithi159.stdout:5431 read 0xe22270 thru 0xe2c5c8 (0xa359 bytes) 2024-09-17T22:58:30.486 INFO:teuthology.orchestra.run.smithi159.stdout:5432 read 0x12fadd2 thru 0x12ffac0 (0x4cef bytes) 2024-09-17T22:58:30.486 INFO:teuthology.orchestra.run.smithi159.stdout:5433 punch from 0x4ba9f9 to 0x4bb417, (0xa1e bytes) 2024-09-17T22:58:30.486 INFO:teuthology.orchestra.run.smithi159.stdout:5434 read 0x2a453b0 thru 0x2a4b572 (0x61c3 bytes) 2024-09-17T22:58:30.489 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:30.486+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:30.502 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.502 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:30.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:30.503 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.233213683999999, 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:30.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:30.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:30.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.232874670000001, 2024-09-17T22:58:30.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:30.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:30.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.513 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 46.055235674999999, 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.514 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:30.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.411045428, 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:58:30.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.411022798999999, 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:30.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:58:30.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.410822713, 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.531 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:30.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.536 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.410797141, 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.537 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:58:30.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:58:30.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.41061609, 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.542 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:30.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:58:30.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.410580661000001, 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:30.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:58:30.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.41048176, 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:58:30.551 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:30.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:58:30.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.410305599999999, 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:58:30.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:58:30.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:58:30.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.410246104, 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:58:30.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.399972934999999, 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:58:30.568 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:58:30.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:58:30.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.399884759000001, 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:58:30.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.577 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.39983178, 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:58:30.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.399773816, 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.582 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:58:30.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.586 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.39948953, 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:58:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.399313347, 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:58:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.398634210000001, 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:58:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:58:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:58:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.398610736, 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:58:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:30.606 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:30.606 INFO:teuthology.orchestra.run.smithi159.stdout:5436 write 0xbe481b2 thru 0xbe52b87 (0xa9d6 bytes) 2024-09-17T22:58:30.606 INFO:teuthology.orchestra.run.smithi159.stdout:5440 trunc from 0xbe52b88 to 0xad3679a 2024-09-17T22:58:30.607 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5441 writesame 0x59b1a9 thru 0x59d8d2 data_size 0x9(0x272a bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5442 writesame 0xd853461 thru 0xd8541d3 data_size 0x1(0xd73 bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5443 punch from 0x62beeb5 to 0x62c08f4, (0x1a3f bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5444 write 0x72efb3c thru 0x72f5f2c (0x63f1 bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5445 writesame 0xea946df thru 0xea9bc8c data_size 0x3(0x75ae bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5447 writesame 0xe5c5e89 thru 0xe5d4a34 data_size 0x2(0xebac bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5449 write 0xdddc780 thru 0xdde8618 (0xbe99 bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5450 writesame 0x73f7c13 thru 0x73ffe97 data_size 0x1(0x8285 bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5454 read 0x1b3fdff thru 0x1b4befc (0xc0fe bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5455 write 0xa87c72a thru 0xa888329 (0xbc00 bytes) 2024-09-17T22:58:30.610 INFO:teuthology.orchestra.run.smithi159.stdout:5456 write 0xc161ce0 thru 0xc168b6e (0x6e8f bytes) 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:5457 trunc from 0xea9bc8d to 0x4e50bd4 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:5458 read 0x4cc8b2 thru 0x4dacb3 (0xe402 bytes) 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:5459 read 0x7a1bce thru 0x7a3d93 (0x21c6 bytes) 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:5460 punch from 0x34234a0 to 0x3432be1, (0xf741 bytes) 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:5461 write 0x59b2830 thru 0x59b3421 (0xbf2 bytes) 2024-09-17T22:58:30.611 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:30.612 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:30.607+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:30.620 INFO:teuthology.orchestra.run.smithi159.stdout:5465 write 0x8c0a101 thru 0x8c0c8cd (0x27cd bytes) 2024-09-17T22:58:30.622 INFO:teuthology.orchestra.run.smithi159.stdout:5466 read 0x68343d thru 0x6902a7 (0xce6b bytes) 2024-09-17T22:58:30.622 INFO:teuthology.orchestra.run.smithi159.stdout:5467 read 0x80fc975 thru 0x8100b21 (0x41ad bytes) 2024-09-17T22:58:30.622 INFO:teuthology.orchestra.run.smithi159.stdout:5468 write 0x5bb67ae thru 0x5bbf43b (0x8c8e bytes) 2024-09-17T22:58:30.632 INFO:teuthology.orchestra.run.smithi159.stdout:5469 trunc from 0x8c0c8ce to 0x15f55f0 2024-09-17T22:58:30.661 INFO:teuthology.orchestra.run.smithi159.stdout:5473 punch from 0x634a75 to 0x64407d, (0xf608 bytes) 2024-09-17T22:58:30.661 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:30.661 INFO:teuthology.orchestra.run.smithi159.stdout:5475 punch from 0xfde28f to 0xfe3253, (0x4fc4 bytes) 2024-09-17T22:58:30.678 INFO:teuthology.orchestra.run.smithi159.stdout:5476 write 0xb54afcc thru 0xb554388 (0x93bd bytes) 2024-09-17T22:58:30.688 INFO:teuthology.orchestra.run.smithi159.stdout:5477 trunc from 0xb554389 to 0x7037a52 2024-09-17T22:58:30.710 INFO:teuthology.orchestra.run.smithi159.stdout:5478 write 0xed7c047 thru 0xed7dfc0 (0x1f7a bytes) 2024-09-17T22:58:30.711 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:30.707+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:30.713 INFO:teuthology.orchestra.run.smithi159.stdout:5480 writesame 0x4fb70b7 thru 0x4fb9da7 data_size 0x1(0x2cf1 bytes) 2024-09-17T22:58:30.720 INFO:teuthology.orchestra.run.smithi159.stdout:5481 write 0x9622335 thru 0x962437c (0x2048 bytes) 2024-09-17T22:58:30.723 INFO:teuthology.orchestra.run.smithi159.stdout:5482 read 0x99c2f0f thru 0x99c3873 (0x965 bytes) 2024-09-17T22:58:30.723 INFO:teuthology.orchestra.run.smithi159.stdout:5483 clone 46 order 20 su 4096 sc 11 2024-09-17T22:58:30.733 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.733 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:30.733 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:30.733 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:30.742 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:30.812 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:30.809+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:30.885 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.885 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:30.886 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:30.886 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:30.886 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:30.894 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:30.939 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:30.937+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:30.999 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:30.999 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958147457000001, 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.000 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:31.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:31.005 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958127041999999, 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.006 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:31.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:31.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:31.011 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958105982999999, 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:31.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958082878000001, 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.017 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:31.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.022 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958059821999999, 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:31.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958029188999999, 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:31.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:31.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.958005213, 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.033 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:31.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:31.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:31.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.039 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.957975035, 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:31.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:31.045 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.957956058000001, 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.046 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:31.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:31.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:31.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:31.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:31.052 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.957517147000001, 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.053 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:31.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.957228541999999, 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:31.058 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.956601638, 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.065 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:31.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:31.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.071 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.956567199, 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:31.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:31.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:31.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.077 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.956179039, 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:31.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:31.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.082 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.954398907, 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.083 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:31.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:31.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.089 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.954384639000001, 2024-09-17T22:58:31.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.091 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:31.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:31.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:31.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:31.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:31.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.097 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.954374183000001, 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.098 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:31.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.954355902, 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.104 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:31.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.953936433999999, 2024-09-17T22:58:31.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:31.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:31.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.114 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.953310189, 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:31.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.119 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.120 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.120 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:31.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:31.122 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:58:31.123 INFO:teuthology.orchestra.run.smithi159.stdout:flattening image image_client.0-clone45 2024-09-17T22:58:31.123 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:31.118+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:31.222 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:31.219+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:31.272 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:31.273 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:31.273 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:31.273 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:31.282 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:31.319 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-17T22:58:31.323 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:31.321+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:31.413 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:31.413 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:31.423 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:31.421+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:31.425 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:31.525 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:31.521+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:31.575 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:31.575 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.418840359999997, 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.576 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:31.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:31.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.067051961000001, 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:31.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:31.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:31.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.067031251, 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.589 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:31.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:31.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:31.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:31.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.595 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.067011455999999, 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.596 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:31.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066992212999999, 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:31.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.605 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066971073000001, 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066945984, 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.610 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:31.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.614 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066926640999998, 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:31.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066904617999999, 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:31.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:31.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:31.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066810477000001, 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.623 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:31.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066783704999999, 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:31.627 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:31.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066763178999999, 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.632 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.635 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066741885999999, 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.636 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:31.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066713011000001, 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.640 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:31.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:31.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:31.644 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066671931999998, 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.649 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.066647909, 2024-09-17T22:58:31.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:31.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:31.655 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.181905282000001, 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.181883654, 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:31.660 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:31.661 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.662 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.663 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.18186247, 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:31.665 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:31.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:31.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.181841390000001, 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:31.670 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:31.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:31.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.674 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:31.675 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:31.675 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:31.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:31.677 INFO:teuthology.orchestra.run.smithi159.stdout:5484 read 0x5d4280d thru 0x5d490aa (0x689e bytes) 2024-09-17T22:58:31.677 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:5486 punch from 0x51718c5 to 0x517b413, (0x9b4e bytes) 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:5487 write 0xe0b8f96 thru 0xe0c5e82 (0xceed bytes) 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:5488 punch from 0x1bcb8b to 0x1c06a9, (0x3b1e bytes) 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:5490 write 0xd4bbe39 thru 0xd4c4052 (0x821a bytes) 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:5495 write 0xbd2fb41 thru 0xbd3afdb (0xb49b bytes) 2024-09-17T22:58:31.678 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:31.673+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:31.690 INFO:teuthology.orchestra.run.smithi159.stdout:5496 clone 47 order 25 su 524288 sc 13 2024-09-17T22:58:31.777 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:31.774+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:31.783 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:31.783 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:31.792 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:31.842 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:31.878 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:31.875+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:31.936 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:31.936 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:31.937 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:31.937 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:31.945 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:31.979 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:31.977+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:32.080 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:32.077+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:32.091 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone46 from 0xed7dfc1 (overlap 0xed7dfc1) to 0x2f7f98d 2024-09-17T22:58:32.096 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:32.096 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89486 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.707405315999999, 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059383219000000001, 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.097 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89486 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:32.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385240+0000", 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385245+0000", 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.639e-06 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385248+0000", 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1319999999999998e-06 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444094+0000", 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058845916999999998 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444207+0000", 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113261 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444621+0000", 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041399299999999998 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.101 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89487 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.707390886000001, 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059624093000000003, 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89487 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.15e-07 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385254+0000", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2389999999999999e-06 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385257+0000", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385260+0000", 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.785e-06 2024-09-17T22:58:32.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444667+0000", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059407142000000003 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444750+0000", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2232999999999997e-05 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444877+0000", 2024-09-17T22:58:32.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012703399999999999 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89488 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 257024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.707380081, 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059823605000000002, 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.106 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89488 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385264+0000", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385265+0000", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.206e-06 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385268+0000", 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-17T22:58:32.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385271+0000", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7389999999999999e-06 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444898+0000", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059626894999999999 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444967+0000", 2024-09-17T22:58:32.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9573999999999995e-05 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445087+0000", 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011978500000000001 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.707369749, 2024-09-17T22:58:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:32.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91012 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668675611999999, 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059095525000000003, 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91012 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.115 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423970+0000", 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423982+0000", 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423988+0000", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482905+0000", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058917737999999997 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482969+0000", 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4054999999999996e-05 2024-09-17T22:58:32.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483063+0000", 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3968999999999994e-05 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91013 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668645425999999, 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059240328000000002, 2024-09-17T22:58:32.119 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91013 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424000+0000", 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T22:58:32.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424012+0000", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1525000000000001e-05 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424018+0000", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483090+0000", 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059072066999999999 2024-09-17T22:58:32.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483131+0000", 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1427000000000002e-05 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483238+0000", 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107161 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91014 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668618743, 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059357291999999999, 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91014 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.124 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:32.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424027+0000", 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424035+0000", 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6389999999999992e-06 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424040+0000", 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9389999999999996e-06 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483266+0000", 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059226144000000001 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483326+0000", 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9531999999999999e-05 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483382+0000", 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6484999999999999e-05 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91015 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668598276000001, 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059570633999999997, 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91015 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424047+0000", 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424050+0000", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5640000000000001e-06 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424053+0000", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483410+0000", 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059356475999999998 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483489+0000", 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9263000000000003e-05 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483616+0000", 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012700299999999999 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.132 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91016 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 231424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668586174, 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059795158000000001, 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91016 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424058+0000", 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0699999999999997e-07 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424060+0000", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8360000000000001e-06 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424068+0000", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424072+0000", 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8249999999999998e-06 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483668+0000", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059595608000000001 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483751+0000", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3778999999999999e-05 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483852+0000", 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101059 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.136 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91017 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668552813, 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059982901999999998, 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91017 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424096+0000", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0710000000000002e-06 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424103+0000", 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424108+0000", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483886+0000", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059777978000000002 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483971+0000", 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5172000000000004e-05 2024-09-17T22:58:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484074+0000", 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102697 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91018 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668527508, 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060220192999999998, 2024-09-17T22:58:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91018 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424124+0000", 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0209999999999999e-06 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424135+0000", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0547000000000001e-05 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424143+0000", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484133+0000", 2024-09-17T22:58:32.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059990452 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484226+0000", 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2367999999999994e-05 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484336+0000", 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011052099999999999 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91019 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668492804, 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060388084000000002, 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91019 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.146 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:58:32.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424153+0000", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2819999999999999e-06 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424162+0000", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424168+0000", 2024-09-17T22:58:32.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484367+0000", 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060199257999999999 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484446+0000", 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9029000000000005e-05 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484539+0000", 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2687000000000002e-05 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91020 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668469093000001, 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060513375000000001, 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.151 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91020 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:32.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424175+0000", 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424177+0000", 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.978e-06 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424183+0000", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1109999999999996e-06 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424190+0000", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2300000000000002e-06 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484556+0000", 2024-09-17T22:58:32.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060365758999999998 2024-09-17T22:58:32.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484614+0000", 2024-09-17T22:58:32.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8233999999999998e-05 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484688+0000", 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3668e-05 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91021 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668444447000001, 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060706860000000001, 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91021 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.157 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424201+0000", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.939e-06 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424207+0000", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424217+0000", 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2e-06 2024-09-17T22:58:32.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484719+0000", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060502242999999997 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484795+0000", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6462000000000001e-05 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484906+0000", 2024-09-17T22:58:32.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011060700000000001 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91022 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668420193999999, 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060885440999999998, 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.161 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91022 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424224+0000", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424226+0000", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424232+0000", 2024-09-17T22:58:32.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1349999999999997e-06 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424240+0000", 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4320000000000002e-06 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484935+0000", 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060695168000000001 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485019+0000", 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4040000000000002e-05 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485109+0000", 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9936000000000001e-05 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668396797, 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:32.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:32.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668376021, 2024-09-17T22:58:32.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668357231, 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:32.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.176 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.177 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:32.178 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.66831202, 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:32.179 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.180 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:32.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:32.182 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.183 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.668278192000001, 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:32.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:32.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:32.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:32.188 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:32.189 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:32.189 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:32.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:32.190 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:32.186+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:32.202 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:32.203 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:32.203 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:32.225 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:32.225 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:32.242 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:32.243 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:32.263 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:32.264 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:32.287 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:32.288 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:32.290 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:32.287+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:32.312 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:32.312 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:32.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:32.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:32.363 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:32.364 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:32.373 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:32.414 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:32.411+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:32.423 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-17T22:58:32.492 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:32.490+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:32.518 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:32.519 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:32.527 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:32.593 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:32.590+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:32.631 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:32.631 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:32.639 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:32.694 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:32.692+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:32.702 INFO:teuthology.orchestra.run.smithi159.stdout:5497 trunc from 0xed7dfc1 to 0x2354f 2024-09-17T22:58:32.765 INFO:teuthology.orchestra.run.smithi159.stdout:5498 trunc from 0x2354f to 0x84447ac 2024-09-17T22:58:32.778 INFO:teuthology.orchestra.run.smithi159.stdout:5499 write 0x981e19f thru 0x982bf7c (0xddde bytes) 2024-09-17T22:58:32.788 INFO:teuthology.orchestra.run.smithi159.stdout:5500 write 0xa7d6cf6 thru 0xa7d90e7 (0x23f2 bytes) 2024-09-17T22:58:32.791 INFO:teuthology.orchestra.run.smithi159.stdout:5501 clone 48 order 21 su 262144 sc 10 2024-09-17T22:58:32.795 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:32.792+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:32.806 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:32.896 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:32.893+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:32.910 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:32.910 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:32.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:32.997 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:32.995+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:33.065 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.065 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.075 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:33.098 INFO:teuthology.orchestra.run.smithi159.stdout:flattening image image_client.0-clone47 2024-09-17T22:58:33.118 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:33.115+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:33.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:58:33.179 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.180 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.188 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:33.198 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:33.196+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:33.229 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-17T22:58:33.300 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:33.297+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:33.383 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.383 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:33.401 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:33.398+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:33.502 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:33.499+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:33.554 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.555 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.563 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:33.603 INFO:teuthology.orchestra.run.smithi159.stdout:5502 writesame 0xb726644 thru 0xb727cc5 data_size 0x2(0x1682 bytes) 2024-09-17T22:58:33.603 INFO:teuthology.orchestra.run.smithi159.stdout:5503 read 0x1a420bb thru 0x1a4d317 (0xb25d bytes) 2024-09-17T22:58:33.604 INFO:teuthology.orchestra.run.smithi159.stdout:5504 read 0xa8665fe thru 0xa8748cd (0xe2d0 bytes) 2024-09-17T22:58:33.604 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:33.604 INFO:teuthology.orchestra.run.smithi159.stdout:5506 read 0xa3e98b5 thru 0xa3ed5de (0x3d2a bytes) 2024-09-17T22:58:33.604 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:33.604 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:33.601+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:33.612 INFO:teuthology.orchestra.run.smithi159.stdout:5508 write 0xe75e865 thru 0xe76a283 (0xba1f bytes) 2024-09-17T22:58:33.616 INFO:teuthology.orchestra.run.smithi159.stdout:5509 read 0xe6bd118 thru 0xe6cad3f (0xdc28 bytes) 2024-09-17T22:58:33.616 INFO:teuthology.orchestra.run.smithi159.stdout:5515 punch from 0xda93903 to 0xda96eac, (0x35a9 bytes) 2024-09-17T22:58:33.616 INFO:teuthology.orchestra.run.smithi159.stdout:5516 read 0xcd1379e thru 0xcd1a1a1 (0x6a04 bytes) 2024-09-17T22:58:33.616 INFO:teuthology.orchestra.run.smithi159.stdout:5518 write 0x8ac232c thru 0x8ad094f (0xe624 bytes) 2024-09-17T22:58:33.619 INFO:teuthology.orchestra.run.smithi159.stdout:5519 write 0x240a584 thru 0x2415b2a (0xb5a7 bytes) 2024-09-17T22:58:33.635 INFO:teuthology.orchestra.run.smithi159.stdout:5521 trunc from 0xe76a284 to 0x35e2855 2024-09-17T22:58:33.660 INFO:teuthology.orchestra.run.smithi159.stdout:5524 write 0x3d6fe27 thru 0x3d79cbd (0x9e97 bytes) 2024-09-17T22:58:33.666 INFO:teuthology.orchestra.run.smithi159.stdout:5525 trunc from 0x3d79cbe to 0x65e7d31 2024-09-17T22:58:33.671 INFO:teuthology.orchestra.run.smithi159.stdout:5526 write 0xf92c77 thru 0xf9691f (0x3ca9 bytes) 2024-09-17T22:58:33.673 INFO:teuthology.orchestra.run.smithi159.stdout:5528 read 0x3385c22 thru 0x3386202 (0x5e1 bytes) 2024-09-17T22:58:33.674 INFO:teuthology.orchestra.run.smithi159.stdout:5529 read 0x1849273 thru 0x184d2b0 (0x403e bytes) 2024-09-17T22:58:33.684 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.685 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.687 INFO:teuthology.orchestra.run.smithi159.stdout:5530 write 0xda1c233 thru 0xda1fad2 (0x38a0 bytes) 2024-09-17T22:58:33.690 INFO:teuthology.orchestra.run.smithi159.stdout:5531 read 0x1c052e2 thru 0x1c11937 (0xc656 bytes) 2024-09-17T22:58:33.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:33.695 INFO:teuthology.orchestra.run.smithi159.stdout:5532 write 0xdad836b thru 0xdadb3ff (0x3095 bytes) 2024-09-17T22:58:33.698 INFO:teuthology.orchestra.run.smithi159.stdout:5535 punch from 0xc69da62 to 0xc6aa429, (0xc9c7 bytes) 2024-09-17T22:58:33.704 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:33.702+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:33.710 INFO:teuthology.orchestra.run.smithi159.stdout:5536 trunc from 0xdadb400 to 0x3d0fe62 2024-09-17T22:58:33.722 INFO:teuthology.orchestra.run.smithi159.stdout:5537 punch from 0x1e4f3be to 0x1e57c7c, (0x88be bytes) 2024-09-17T22:58:33.740 INFO:teuthology.orchestra.run.smithi159.stdout:5539 writesame 0xdb1c2af thru 0xdb285b5 data_size 0x1(0xc307 bytes) 2024-09-17T22:58:33.740 INFO:teuthology.orchestra.run.smithi159.stdout:5540 read 0x901b4c0 thru 0x9028e31 (0xd972 bytes) 2024-09-17T22:58:33.740 INFO:teuthology.orchestra.run.smithi159.stdout:5542 write 0x9021b5a thru 0x902daa4 (0xbf4b bytes) 2024-09-17T22:58:33.742 INFO:teuthology.orchestra.run.smithi159.stdout:5543 read 0x34cdd30 thru 0x34da182 (0xc453 bytes) 2024-09-17T22:58:33.747 INFO:teuthology.orchestra.run.smithi159.stdout:5544 trunc from 0xdb285b6 to 0xa3e3722 2024-09-17T22:58:33.749 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:58:33.749 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:33.749 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T22:58:33.749 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:58:33.749 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:58:33.750 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T22:58:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:5545 write 0x2b2e268 thru 0x2b32d9e (0x4b37 bytes) 2024-09-17T22:58:33.753 INFO:teuthology.orchestra.run.smithi159.stdout:5547 read 0x102b94c thru 0x102c458 (0xb0d bytes) 2024-09-17T22:58:33.754 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:33.754 INFO:teuthology.orchestra.run.smithi159.stdout:5549 writesame 0x26d95fa thru 0x26dff07 data_size 0x71(0x690e bytes) 2024-09-17T22:58:33.757 INFO:teuthology.orchestra.run.smithi159.stdout:5550 read 0x605163c thru 0x6057da8 (0x676d bytes) 2024-09-17T22:58:33.765 INFO:teuthology.orchestra.run.smithi159.stdout:5552 write 0xd8f17cf thru 0xd8f286c (0x109e bytes) 2024-09-17T22:58:33.768 INFO:teuthology.orchestra.run.smithi159.stdout:5554 read 0xcff949 thru 0xd083cd (0x8a85 bytes) 2024-09-17T22:58:33.779 INFO:teuthology.orchestra.run.smithi159.stdout:5555 trunc from 0xd8f286d to 0x5172897 2024-09-17T22:58:33.789 INFO:teuthology.orchestra.run.smithi159.stdout:5556 read 0x46f26f1 thru 0x4700b39 (0xe449 bytes) 2024-09-17T22:58:33.789 INFO:teuthology.orchestra.run.smithi159.stdout:5557 writesame 0x2e7e481 thru 0x2e8bc22 data_size 0x7(0xd7a2 bytes) 2024-09-17T22:58:33.794 INFO:teuthology.orchestra.run.smithi159.stdout:5558 read 0x56fa9d thru 0x57b19f (0xb703 bytes) 2024-09-17T22:58:33.797 INFO:teuthology.orchestra.run.smithi159.stdout:5559 trunc from 0x5172897 to 0x242a935 2024-09-17T22:58:33.804 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:33.801+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:33.819 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:33.825 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.825 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.826 INFO:teuthology.orchestra.run.smithi159.stdout:5562 trunc from 0x242a935 to 0x8203f03 2024-09-17T22:58:33.831 INFO:teuthology.orchestra.run.smithi159.stdout:5563 writesame 0x2f47ffe thru 0x2f55af2 data_size 0x1(0xdaf5 bytes) 2024-09-17T22:58:33.836 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:33.857 INFO:teuthology.orchestra.run.smithi159.stdout:5565 punch from 0x41cc710 to 0x41dc3d9, (0xfcc9 bytes) 2024-09-17T22:58:33.857 INFO:teuthology.orchestra.run.smithi159.stdout:5566 punch from 0x297b1b8 to 0x2989054, (0xde9c bytes) 2024-09-17T22:58:33.867 INFO:teuthology.orchestra.run.smithi159.stdout:5567 writesame 0xcb15c34 thru 0xcb19a17 data_size 0x22(0x3de4 bytes) 2024-09-17T22:58:33.868 INFO:teuthology.orchestra.run.smithi159.stdout:5568 read 0x2a66ae4 thru 0x2a69112 (0x262f bytes) 2024-09-17T22:58:33.874 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:58:33.874 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:33.879 INFO:teuthology.orchestra.run.smithi159.stdout:5569 trunc from 0xcb19a18 to 0x3915495 2024-09-17T22:58:33.889 INFO:teuthology.orchestra.run.smithi159.stdout:5570 trunc from 0x3915495 to 0x1a271f5 2024-09-17T22:58:33.902 INFO:teuthology.orchestra.run.smithi159.stdout:5571 trunc from 0x1a271f5 to 0x30af8e 2024-09-17T22:58:33.905 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:33.902+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:33.924 INFO:teuthology.orchestra.run.smithi159.stdout:5573 punch from 0x308af2 to 0x30af8e, (0x249c bytes) 2024-09-17T22:58:33.924 INFO:teuthology.orchestra.run.smithi159.stdout:5574 read 0x1d11d1 thru 0x1de833 (0xd663 bytes) 2024-09-17T22:58:33.924 INFO:teuthology.orchestra.run.smithi159.stdout:5575 punch from 0xcbd86 to 0xcbe76, (0xf0 bytes) 2024-09-17T22:58:33.949 INFO:teuthology.orchestra.run.smithi159.stdout:5576 write 0xe488e21 thru 0xe491365 (0x8545 bytes) 2024-09-17T22:58:33.951 INFO:teuthology.orchestra.run.smithi159.stdout:5577 read 0xc432f2 thru 0xc52d2f (0xfa3e bytes) 2024-09-17T22:58:33.952 INFO:teuthology.orchestra.run.smithi159.stdout:5578 read 0x261df29 thru 0x262333b (0x5413 bytes) 2024-09-17T22:58:33.952 INFO:teuthology.orchestra.run.smithi159.stdout:5579 write 0xe36d16f thru 0xe36d866 (0x6f8 bytes) 2024-09-17T22:58:33.954 INFO:teuthology.orchestra.run.smithi159.stdout:5580 read 0x8bbcf2 thru 0x8c2913 (0x6c22 bytes) 2024-09-17T22:58:33.955 INFO:teuthology.orchestra.run.smithi159.stdout:5581 punch from 0x6c38a4f to 0x6c404f5, (0x7aa6 bytes) 2024-09-17T22:58:33.955 INFO:teuthology.orchestra.run.smithi159.stdout:5583 write 0x8185046 thru 0x8192943 (0xd8fe bytes) 2024-09-17T22:58:33.958 INFO:teuthology.orchestra.run.smithi159.stdout:5584 read 0x8c1df82 thru 0x8c2cb83 (0xec02 bytes) 2024-09-17T22:58:33.959 INFO:teuthology.orchestra.run.smithi159.stdout:5585 read 0x463a397 thru 0x4649fc1 (0xfc2b bytes) 2024-09-17T22:58:33.959 INFO:teuthology.orchestra.run.smithi159.stdout:5586 punch from 0x89d8ea1 to 0x89dc0f8, (0x3257 bytes) 2024-09-17T22:58:33.959 INFO:teuthology.orchestra.run.smithi159.stdout:5587 read 0x5657836 thru 0x5660a48 (0x9213 bytes) 2024-09-17T22:58:33.959 INFO:teuthology.orchestra.run.smithi159.stdout:5588 write 0x41a219e thru 0x41b1d18 (0xfb7b bytes) 2024-09-17T22:58:33.961 INFO:teuthology.orchestra.run.smithi159.stdout:5591 punch from 0xd52976 to 0xd59f43, (0x75cd bytes) 2024-09-17T22:58:33.961 INFO:teuthology.orchestra.run.smithi159.stdout:5592 punch from 0x8669180 to 0x8669204, (0x84 bytes) 2024-09-17T22:58:33.962 INFO:teuthology.orchestra.run.smithi159.stdout:5593 writesame 0xd3b89e8 thru 0xd3ba8d1 data_size 0x3(0x1eea bytes) 2024-09-17T22:58:33.964 INFO:teuthology.orchestra.run.smithi159.stdout:5594 read 0xa43e87f thru 0xa44645c (0x7bde bytes) 2024-09-17T22:58:33.965 INFO:teuthology.orchestra.run.smithi159.stdout:5597 writesame 0x21d47ff thru 0x21d6e84 data_size 0x1(0x2686 bytes) 2024-09-17T22:58:33.971 INFO:teuthology.orchestra.run.smithi159.stdout:5599 clone 49 order 24 su 131072 sc 9 2024-09-17T22:58:33.990 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:33.990 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:33.999 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:34.006 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:34.003+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:34.039 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:58:34.097 INFO:teuthology.orchestra.run.smithi159.stdout:leaving image image_client.0-clone48 intact 2024-09-17T22:58:34.107 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:34.104+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:34.151 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:34.152 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:34.161 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:34.207 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:34.204+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:34.251 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-17T22:58:34.308 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:34.306+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:34.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:34.330 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:34.330 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:34.340 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:34.425 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:34.423+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:34.442 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.442 INFO:teuthology.orchestra.run.smithi159.stdout:5601 write 0xc68937d thru 0xc694267 (0xaeeb bytes) 2024-09-17T22:58:34.445 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:34.445 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:34.449 INFO:teuthology.orchestra.run.smithi159.stdout:5602 read 0x73ccee3 thru 0x73d8f09 (0xc027 bytes) 2024-09-17T22:58:34.450 INFO:teuthology.orchestra.run.smithi159.stdout:5603 read 0x1177af6 thru 0x117a972 (0x2e7d bytes) 2024-09-17T22:58:34.450 INFO:teuthology.orchestra.run.smithi159.stdout:5604 read 0xe414ee5 thru 0xe419c6d (0x4d89 bytes) 2024-09-17T22:58:34.450 INFO:teuthology.orchestra.run.smithi159.stdout:5605 read 0xa4efb1c thru 0xa4f140d (0x18f2 bytes) 2024-09-17T22:58:34.450 INFO:teuthology.orchestra.run.smithi159.stdout:5606 write 0xa3d2665 thru 0xa3dc62e (0x9fca bytes) 2024-09-17T22:58:34.454 INFO:teuthology.orchestra.run.smithi159.stdout:5607 read 0xbf232f6 thru 0xbf26409 (0x3114 bytes) 2024-09-17T22:58:34.454 INFO:teuthology.orchestra.run.smithi159.stdout:5608 read 0xc42c676 thru 0xc43613c (0x9ac7 bytes) 2024-09-17T22:58:34.454 INFO:teuthology.orchestra.run.smithi159.stdout:5609 read 0x3c24584 thru 0x3c31329 (0xcda6 bytes) 2024-09-17T22:58:34.454 INFO:teuthology.orchestra.run.smithi159.stdout:5610 read 0xa72c5ad thru 0xa73450a (0x7f5e bytes) 2024-09-17T22:58:34.454 INFO:teuthology.orchestra.run.smithi159.stdout:5611 write 0x8504bea thru 0x8512bea (0xe001 bytes) 2024-09-17T22:58:34.459 INFO:teuthology.orchestra.run.smithi159.stdout:5612 read 0xbd9d10a thru 0xbd9e400 (0x12f7 bytes) 2024-09-17T22:58:34.459 INFO:teuthology.orchestra.run.smithi159.stdout:5613 write 0x920f3fa thru 0x9211284 (0x1e8b bytes) 2024-09-17T22:58:34.466 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:34.466 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:34.467 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:34.468 INFO:teuthology.orchestra.run.smithi159.stdout:5614 trunc from 0xe491366 to 0xa3565e2 2024-09-17T22:58:34.488 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:34.488 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:34.489 INFO:teuthology.orchestra.run.smithi159.stdout:5615 write 0xb09e868 thru 0xb0a7353 (0x8aec bytes) 2024-09-17T22:58:34.495 INFO:teuthology.orchestra.run.smithi159.stdout:5616 trunc from 0xb0a7354 to 0x6c6807b 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:34.506 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463825023999998, 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:34.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:34.511 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463804609, 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:34.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463783549999999, 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.516 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:34.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:34.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463760444999998, 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:34.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463737388999999, 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:34.526 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:34.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.530 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463706756000001, 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:34.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463682779999999, 2024-09-17T22:58:34.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.537 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:34.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:34.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:34.542 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463652602, 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:34.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:34.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463633625, 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.547 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:34.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.463194714, 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.552 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:34.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.556 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.462906108999999, 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:34.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.462279205000002, 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.561 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.462244766000001, 2024-09-17T22:58:34.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:34.570 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.461856606000001, 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.571 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:34.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.575 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.460076474000001, 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:34.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:34.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.460062206, 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.580 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.460051750000002, 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:34.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:34.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:34.589 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.460033468999999, 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:34.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.459614000999998, 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:34.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 18.458987755999999, 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:34.599 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:34.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:34.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.603 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.604 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.604 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:34.604 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:34.604 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:34.605 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:34.605 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:34.602+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:34.605 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:34.606 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:34.607 INFO:teuthology.orchestra.run.smithi159.stdout:5618 read 0x29c39b2 thru 0x29d2309 (0xe958 bytes) 2024-09-17T22:58:34.607 INFO:teuthology.orchestra.run.smithi159.stdout:5620 write 0x13b304a thru 0x13b4080 (0x1037 bytes) 2024-09-17T22:58:34.607 INFO:teuthology.orchestra.run.smithi159.stdout:5621 write 0x8612e01 thru 0x861fae8 (0xcce8 bytes) 2024-09-17T22:58:34.607 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:34.608 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:34.608 INFO:teuthology.orchestra.run.smithi159.stdout:5622 write 0x9a6b471 thru 0x9a717e7 (0x6377 bytes) 2024-09-17T22:58:34.608 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.609 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:34.609 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:34.609 INFO:teuthology.orchestra.run.smithi159.stdout:5624 trunc from 0x9a717e8 to 0x1300c16 2024-09-17T22:58:34.609 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:34.610 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:34.610 INFO:teuthology.orchestra.run.smithi159.stdout:5625 trunc from 0x1300c16 to 0xd00844d 2024-09-17T22:58:34.610 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:34.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:34.611 INFO:teuthology.orchestra.run.smithi159.stdout:5627 writesame 0xc2a7b34 thru 0xc2b1a98 data_size 0x1(0x9f65 bytes) 2024-09-17T22:58:34.611 INFO:teuthology.orchestra.run.smithi159.stdout:5628 trunc from 0xd00844d to 0xcc686e6 2024-09-17T22:58:34.613 INFO:teuthology.orchestra.run.smithi159.stdout:5630 write 0xdfa6f97 thru 0xdfae67a (0x76e4 bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:5634 punch from 0xba6fc31 to 0xba781e7, (0x85b6 bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:5635 punch from 0x81a520 to 0x828df4, (0xe8d4 bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:5637 punch from 0x4930765 to 0x493690e, (0x61a9 bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:5638 punch from 0xce393d7 to 0xce3ea65, (0x568e bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:5639 punch from 0x27ba449 to 0x27c4aa5, (0xa65c bytes) 2024-09-17T22:58:34.616 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5642 read 0x7cb1ea0 thru 0x7cb2cfd (0xe5e bytes) 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5643 read 0x3a8e63a thru 0x3a93dce (0x5795 bytes) 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5644 read 0x18b4068 thru 0x18c14fb (0xd494 bytes) 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5645 punch from 0x8161d9e to 0x816725d, (0x54bf bytes) 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5646 read 0xcda1d0d thru 0xcda266c (0x960 bytes) 2024-09-17T22:58:34.617 INFO:teuthology.orchestra.run.smithi159.stdout:5647 write 0x2c84ee8 thru 0x2c90f86 (0xc09f bytes) 2024-09-17T22:58:34.618 INFO:teuthology.orchestra.run.smithi159.stdout:5649 write 0x694ca0c thru 0x6954455 (0x7a4a bytes) 2024-09-17T22:58:34.619 INFO:teuthology.orchestra.run.smithi159.stdout:5650 read 0xf46948 thru 0xf48503 (0x1bbc bytes) 2024-09-17T22:58:34.620 INFO:teuthology.orchestra.run.smithi159.stdout:5651 read 0x14b948b thru 0x14c7770 (0xe2e6 bytes) 2024-09-17T22:58:34.620 INFO:teuthology.orchestra.run.smithi159.stdout:5654 write 0x19a08d2 thru 0x19a7268 (0x6997 bytes) 2024-09-17T22:58:34.622 INFO:teuthology.orchestra.run.smithi159.stdout:5655 punch from 0x2226259 to 0x223074a, (0xa4f1 bytes) 2024-09-17T22:58:34.622 INFO:teuthology.orchestra.run.smithi159.stdout:5656 write 0xdf08b5c thru 0xdf0a29d (0x1742 bytes) 2024-09-17T22:58:34.625 INFO:teuthology.orchestra.run.smithi159.stdout:5659 punch from 0x520aab2 to 0x5216bb8, (0xc106 bytes) 2024-09-17T22:58:34.630 INFO:teuthology.orchestra.run.smithi159.stdout:5660 writesame 0xe2973ea thru 0xe29f266 data_size 0x1(0x7e7d bytes) 2024-09-17T22:58:34.636 INFO:teuthology.orchestra.run.smithi159.stdout:5662 write 0xe8c21d5 thru 0xe8ca703 (0x852f bytes) 2024-09-17T22:58:34.639 INFO:teuthology.orchestra.run.smithi159.stdout:5663 write 0x5c61cf3 thru 0x5c63259 (0x1567 bytes) 2024-09-17T22:58:34.640 INFO:teuthology.orchestra.run.smithi159.stdout:5664 read 0xbf23fb thru 0xbf7500 (0x5106 bytes) 2024-09-17T22:58:34.641 INFO:teuthology.orchestra.run.smithi159.stdout:5665 writesame 0x9ed590e thru 0x9edc1ab data_size 0x7(0x689e bytes) 2024-09-17T22:58:34.645 INFO:teuthology.orchestra.run.smithi159.stdout:5666 write 0x930151b thru 0x93080df (0x6bc5 bytes) 2024-09-17T22:58:34.648 INFO:teuthology.orchestra.run.smithi159.stdout:5667 read 0xba3cc7f thru 0xba40d9c (0x411e bytes) 2024-09-17T22:58:34.651 INFO:teuthology.orchestra.run.smithi159.stdout:5668 trunc from 0xe8ca704 to 0xc029df8 2024-09-17T22:58:34.658 INFO:teuthology.orchestra.run.smithi159.stdout:5669 writesame 0x8772f5e thru 0x8778a41 data_size 0x1c(0x5ae4 bytes) 2024-09-17T22:58:34.659 INFO:teuthology.orchestra.run.smithi159.stdout:5670 read 0x69fe223 thru 0x6a05a37 (0x7815 bytes) 2024-09-17T22:58:34.660 INFO:teuthology.orchestra.run.smithi159.stdout:5672 read 0x4bdc294 thru 0x4be9bee (0xd95b bytes) 2024-09-17T22:58:34.667 INFO:teuthology.orchestra.run.smithi159.stdout:5673 write 0xd95c22f thru 0xd96a70d (0xe4df bytes) 2024-09-17T22:58:34.669 INFO:teuthology.orchestra.run.smithi159.stdout:5674 read 0x3649e50 thru 0x364bf3f (0x20f0 bytes) 2024-09-17T22:58:34.669 INFO:teuthology.orchestra.run.smithi159.stdout:5677 read 0x715965c thru 0x716055b (0x6f00 bytes) 2024-09-17T22:58:34.670 INFO:teuthology.orchestra.run.smithi159.stdout:5678 punch from 0x196bfc4 to 0x196dd79, (0x1db5 bytes) 2024-09-17T22:58:34.670 INFO:teuthology.orchestra.run.smithi159.stdout:5679 writesame 0xbe185cc thru 0xbe1f3b0 data_size 0x1(0x6de5 bytes) 2024-09-17T22:58:34.682 INFO:teuthology.orchestra.run.smithi159.stdout:5680 write 0x1afd5ca thru 0x1b0d0d6 (0xfb0d bytes) 2024-09-17T22:58:34.684 INFO:teuthology.orchestra.run.smithi159.stdout:5681 writesame 0x809a2fc thru 0x80a0b14 data_size 0x3(0x6819 bytes) 2024-09-17T22:58:34.688 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.688 INFO:teuthology.orchestra.run.smithi159.stdout:5686 punch from 0x8ce4d3a to 0x8ce836f, (0x3635 bytes) 2024-09-17T22:58:34.688 INFO:teuthology.orchestra.run.smithi159.stdout:5687 write 0xd5df8af thru 0xd5e6a77 (0x71c9 bytes) 2024-09-17T22:58:34.690 INFO:teuthology.orchestra.run.smithi159.stdout:5688 read 0xafeacff thru 0xaff5d87 (0xb089 bytes) 2024-09-17T22:58:34.690 INFO:teuthology.orchestra.run.smithi159.stdout:5690 punch from 0x874210 to 0x874bef, (0x9df bytes) 2024-09-17T22:58:34.699 INFO:teuthology.orchestra.run.smithi159.stdout:5691 trunc from 0xd96a70e to 0x5fd5ce6 2024-09-17T22:58:34.945 INFO:teuthology.orchestra.run.smithi159.stdout:5692 write 0x56721a8 thru 0x567f946 (0xd79f bytes) 2024-09-17T22:58:34.945 INFO:teuthology.orchestra.run.smithi159.stdout:5693 read 0x4e7bdc8 thru 0x4e834c9 (0x7702 bytes) 2024-09-17T22:58:34.945 INFO:teuthology.orchestra.run.smithi159.stdout:5694 writesame 0x302a0b7 thru 0x30365a0 data_size 0x47(0xc4ea bytes) 2024-09-17T22:58:34.945 INFO:teuthology.orchestra.run.smithi159.stdout:5695 write 0x204c0f8 thru 0x205296d (0x6876 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5697 write 0x873babf thru 0x8747d65 (0xc2a7 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5700 punch from 0x4657bb6 to 0x465e63f, (0x6a89 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5703 writesame 0x3252835 thru 0x325a861 data_size 0x13(0x802d bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5705 write 0x50f37d5 thru 0x5102ea8 (0xf6d4 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5709 read 0x672d7fc thru 0x6734720 (0x6f25 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:5710 read 0x39d3aee thru 0x39dc733 (0x8c46 bytes) 2024-09-17T22:58:34.946 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:34.947 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:34.947 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:34.947 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:34.947 INFO:teuthology.orchestra.run.smithi159.stdout:5711 read 0x560ab39 thru 0x561422d (0x96f5 bytes) 2024-09-17T22:58:34.947 INFO:teuthology.orchestra.run.smithi159.stdout:5712 write 0x97fc9e1 thru 0x9804ba6 (0x81c6 bytes) 2024-09-17T22:58:34.948 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:34.950 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:34.942+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:34.955 INFO:teuthology.orchestra.run.smithi159.stdout:5713 writesame 0x33b3fc2 thru 0x33b928b data_size 0x1(0x52ca bytes) 2024-09-17T22:58:34.955 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5720 read 0x2a3a102 thru 0x2a489c7 (0xe8c6 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5721 read 0x7a6bcef thru 0x7a79d03 (0xe015 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5722 punch from 0x5ad759c to 0x5ae68e3, (0xf347 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5723 read 0x19290e0 thru 0x1938c4e (0xfb6f bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5724 read 0xb32cc1 thru 0xb41550 (0xe890 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5728 read 0x3236596 thru 0x3244640 (0xe0ab bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5729 read 0x6007e12 thru 0x60139be (0xbbad bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5730 write 0x6e87b20 thru 0x6e90f24 (0x9405 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5731 write 0x5724704 thru 0x572bd5a (0x7657 bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5732 writesame 0x14c28c7 thru 0x14cf7d1 data_size 0x1(0xcf0b bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5734 read 0x259e679 thru 0x25a0e33 (0x27bb bytes) 2024-09-17T22:58:34.956 INFO:teuthology.orchestra.run.smithi159.stdout:5735 read 0x6860a2b thru 0x68691e0 (0x87b6 bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:5741 read 0x530ff10 thru 0x531a0bd (0xa1ae bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:5742 write 0x54911ac thru 0x549c7ea (0xb63f bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:5743 write 0x7151539 thru 0x716059e (0xf066 bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:5747 write 0xe7a1de3 thru 0xe7aad9a (0x8fb8 bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:5748 write 0xb83d60b thru 0xb84a680 (0xd076 bytes) 2024-09-17T22:58:34.957 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5752 trunc from 0xe7aad9b to 0x5eb99d 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5753 read 0x5064e1 thru 0x516119 (0xfc39 bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5754 write 0xa978977 thru 0xa97c41d (0x3aa7 bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5755 read 0x4550831 thru 0x45607ed (0xffbd bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5760 trunc from 0xa97c41e to 0x25c43fc 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5761 write 0x176dac4 thru 0x1770364 (0x28a1 bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5764 read 0x21c09c0 thru 0x21cf519 (0xeb5a bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5765 trunc from 0x25c43fc to 0xb70273d 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5768 write 0xb96c420 thru 0xb97ab7c (0xe75d bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5769 writesame 0x1e74ae6 thru 0x1e7b79f data_size 0x3(0x6cba bytes) 2024-09-17T22:58:34.958 INFO:teuthology.orchestra.run.smithi159.stdout:5770 read 0xe8db49 thru 0xe9903e (0xb4f6 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5771 read 0x4e46001 thru 0x4e51b17 (0xbb17 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5772 read 0xa16e1b6 thru 0xa17cf76 (0xedc1 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5773 read 0x279fd92 thru 0x27ad525 (0xd794 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5774 trunc from 0xb97ab7d to 0xd6a5fd3 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5776 read 0x2d8eae9 thru 0x2d9d605 (0xeb1d bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5777 write 0x71594d9 thru 0x715ae38 (0x1960 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5778 read 0x6c6a148 thru 0x6c76701 (0xc5ba bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5779 read 0x8cb750 thru 0x8cf1af (0x3a60 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5780 read 0x4119eb7 thru 0x411b0f6 (0x1240 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5781 read 0x3327729 thru 0x332b280 (0x3b58 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5782 writesame 0x217138a thru 0x217c8ed data_size 0x5e(0xb564 bytes) 2024-09-17T22:58:34.959 INFO:teuthology.orchestra.run.smithi159.stdout:5784 write 0x1aea56f thru 0x1aed026 (0x2ab8 bytes) 2024-09-17T22:58:34.960 INFO:teuthology.orchestra.run.smithi159.stdout:5785 read 0x4845b80 thru 0x48537ad (0xdc2e bytes) 2024-09-17T22:58:34.960 INFO:teuthology.orchestra.run.smithi159.stdout:5787 punch from 0x891acca to 0x891bb0f, (0xe45 bytes) 2024-09-17T22:58:34.960 INFO:teuthology.orchestra.run.smithi159.stdout:5789 writesame 0xca40414 thru 0xca430c5 data_size 0x3(0x2cb2 bytes) 2024-09-17T22:58:34.960 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.960 INFO:teuthology.orchestra.run.smithi159.stdout:5792 writesame 0x2c1a382 thru 0x2c1c290 data_size 0x1(0x1f0f bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5794 read 0x2edc39d thru 0x2edf27d (0x2ee1 bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5797 read 0x32a83a6 thru 0x32a9ed1 (0x1b2c bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5798 read 0x956bbf0 thru 0x956d9c5 (0x1dd6 bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5799 read 0x8086965 thru 0x8089385 (0x2a21 bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5800 write 0xbd464f0 thru 0xbd4adc7 (0x48d8 bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5801 writesame 0x55518da thru 0x555c9d6 data_size 0xb(0xb0fd bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5802 writesame 0x8ad34a2 thru 0x8adc075 data_size 0x13(0x8bd4 bytes) 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.961 INFO:teuthology.orchestra.run.smithi159.stdout:5805 writesame 0x2c703c0 thru 0x2c7b93e data_size 0x1(0xb57f bytes) 2024-09-17T22:58:34.962 INFO:teuthology.orchestra.run.smithi159.stdout:5806 read 0x57b2da2 thru 0x57b76e1 (0x4940 bytes) 2024-09-17T22:58:34.962 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:34.962 INFO:teuthology.orchestra.run.smithi159.stdout:5810 read 0x7017218 thru 0x7020c69 (0x9a52 bytes) 2024-09-17T22:58:34.971 INFO:teuthology.orchestra.run.smithi159.stdout:5811 trunc from 0xd6a5fd3 to 0x2a713cd 2024-09-17T22:58:34.989 INFO:teuthology.orchestra.run.smithi159.stdout:5812 writesame 0x32e1397 thru 0x32e9fa2 data_size 0x2(0x8c0c bytes) 2024-09-17T22:58:34.990 INFO:teuthology.orchestra.run.smithi159.stdout:5813 punch from 0x80a11a to 0x817e10, (0xdcf6 bytes) 2024-09-17T22:58:35.004 INFO:teuthology.orchestra.run.smithi159.stdout:5814 write 0x8d94d72 thru 0x8da220f (0xd49e bytes) 2024-09-17T22:58:35.005 INFO:teuthology.orchestra.run.smithi159.stdout:5815 write 0x7417bbf thru 0x74277ba (0xfbfc bytes) 2024-09-17T22:58:35.007 INFO:teuthology.orchestra.run.smithi159.stdout:5816 write 0x6f8f3a9 thru 0x6f9a57c (0xb1d4 bytes) 2024-09-17T22:58:35.008 INFO:teuthology.orchestra.run.smithi159.stdout:5818 read 0x6a166cc thru 0x6a16bc3 (0x4f8 bytes) 2024-09-17T22:58:35.008 INFO:teuthology.orchestra.run.smithi159.stdout:5819 read 0x203ef3b thru 0x2047e5e (0x8f24 bytes) 2024-09-17T22:58:35.009 INFO:teuthology.orchestra.run.smithi159.stdout:5820 read 0x22a2deb thru 0x22aac97 (0x7ead bytes) 2024-09-17T22:58:35.018 INFO:teuthology.orchestra.run.smithi159.stdout:5821 writesame 0xbd26f1b thru 0xbd27422 data_size 0xe(0x508 bytes) 2024-09-17T22:58:35.019 INFO:teuthology.orchestra.run.smithi159.stdout:5822 write 0x57c687 thru 0x58c10c (0xfa86 bytes) 2024-09-17T22:58:35.020 INFO:teuthology.orchestra.run.smithi159.stdout:5823 write 0x5153a80 thru 0x51565de (0x2b5f bytes) 2024-09-17T22:58:35.028 INFO:teuthology.orchestra.run.smithi159.stdout:5824 write 0xc770669 thru 0xc776a1a (0x63b2 bytes) 2024-09-17T22:58:35.031 INFO:teuthology.orchestra.run.smithi159.stdout:5825 write 0x14e04b2 thru 0x14eb65e (0xb1ad bytes) 2024-09-17T22:58:35.032 INFO:teuthology.orchestra.run.smithi159.stdout:5827 write 0xa19a67a thru 0xa1a41ca (0x9b51 bytes) 2024-09-17T22:58:35.035 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:35.035 INFO:teuthology.orchestra.run.smithi159.stdout:5830 write 0xa0acac8 thru 0xa0b5970 (0x8ea9 bytes) 2024-09-17T22:58:35.038 INFO:teuthology.orchestra.run.smithi159.stdout:5832 read 0xbd00296 thru 0xbd02aa6 (0x2811 bytes) 2024-09-17T22:58:35.038 INFO:teuthology.orchestra.run.smithi159.stdout:5833 trunc from 0xc776a1b to 0xce72df5 2024-09-17T22:58:35.044 INFO:teuthology.orchestra.run.smithi159.stdout:5834 writesame 0x536f91e thru 0x5373088 data_size 0x1(0x376b bytes) 2024-09-17T22:58:35.045 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:35.042+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:35.052 INFO:teuthology.orchestra.run.smithi159.stdout:5835 read 0x5d8ade6 thru 0x5d8f7cb (0x49e6 bytes) 2024-09-17T22:58:35.052 INFO:teuthology.orchestra.run.smithi159.stdout:5836 writesame 0x28a3672 thru 0x28a80a6 data_size 0xb(0x4a35 bytes) 2024-09-17T22:58:35.054 INFO:teuthology.orchestra.run.smithi159.stdout:5837 clone 50 order 20 su 4096 sc 9 2024-09-17T22:58:35.078 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:35.078 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:35.078 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:35.079 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:35.079 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:35.087 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:35.171 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:35.169+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:35.232 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:35.232 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:35.232 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:35.232 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:35.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 36.075584747999997, 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.233 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:35.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.237 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:35.239 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.240 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723796349000001, 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.241 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.245 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723775638999999, 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:35.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:35.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723755843999999, 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.250 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:35.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.253 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:35.254 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723736600999999, 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.255 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723715461000001, 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723690372, 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723671028999998, 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723649005999999, 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723554865000001, 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723528092999999, 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723507566999999, 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723486273999999, 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723457399000001, 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723416319999998, 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 21.723392297, 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.838649669999999, 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.838628042, 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.838606857999999, 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.838585777999999, 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:35.329 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:35.331 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:35.331 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:35.328+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:35.432 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:35.428+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:35.435 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:35.435 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:35.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:35.435 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:35.444 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:35.533 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:35.530+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:35.588 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:35.588 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:35.589 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:35.597 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:35.634 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:35.632+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:35.735 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:35.732+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89486 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.354674712000001, 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059383219000000001, 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89486 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385240+0000", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385245+0000", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.639e-06 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385248+0000", 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1319999999999998e-06 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444094+0000", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058845916999999998 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444207+0000", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113261 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444621+0000", 2024-09-17T22:58:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041399299999999998 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89487 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.354660282000001, 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059624093000000003, 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89487 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.15e-07 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385254+0000", 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2389999999999999e-06 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385257+0000", 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385260+0000", 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.785e-06 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444667+0000", 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059407142000000003 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444750+0000", 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2232999999999997e-05 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444877+0000", 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012703399999999999 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89488 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 257024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.354649476999999, 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059823605000000002, 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89488 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385264+0000", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385265+0000", 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.206e-06 2024-09-17T22:58:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385268+0000", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385271+0000", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7389999999999999e-06 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444898+0000", 2024-09-17T22:58:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059626894999999999 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444967+0000", 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9573999999999995e-05 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445087+0000", 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011978500000000001 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.354639145, 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91012 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315945008, 2024-09-17T22:58:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059095525000000003, 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91012 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423970+0000", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423982+0000", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423988+0000", 2024-09-17T22:58:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482905+0000", 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058917737999999997 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482969+0000", 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4054999999999996e-05 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483063+0000", 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3968999999999994e-05 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91013 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315914822, 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059240328000000002, 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91013 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424000+0000", 2024-09-17T22:58:35.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424012+0000", 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1525000000000001e-05 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424018+0000", 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483090+0000", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059072066999999999 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483131+0000", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1427000000000002e-05 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483238+0000", 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107161 2024-09-17T22:58:35.771 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91014 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315888138999998, 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059357291999999999, 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.772 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91014 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:35.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424027+0000", 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424035+0000", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6389999999999992e-06 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424040+0000", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9389999999999996e-06 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483266+0000", 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059226144000000001 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483326+0000", 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9531999999999999e-05 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483382+0000", 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6484999999999999e-05 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91015 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315867672, 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059570633999999997, 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91015 2024-09-17T22:58:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424047+0000", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424050+0000", 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5640000000000001e-06 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424053+0000", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483410+0000", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059356475999999998 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483489+0000", 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9263000000000003e-05 2024-09-17T22:58:35.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483616+0000", 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012700299999999999 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91016 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 231424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.31585557, 2024-09-17T22:58:35.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059795158000000001, 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91016 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424058+0000", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0699999999999997e-07 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424060+0000", 2024-09-17T22:58:35.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8360000000000001e-06 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424068+0000", 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424072+0000", 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8249999999999998e-06 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.784 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483668+0000", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059595608000000001 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483751+0000", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3778999999999999e-05 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483852+0000", 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101059 2024-09-17T22:58:35.785 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91017 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315822209, 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059982901999999998, 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.786 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91017 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424096+0000", 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0710000000000002e-06 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424103+0000", 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424108+0000", 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483886+0000", 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059777978000000002 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483971+0000", 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5172000000000004e-05 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484074+0000", 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102697 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91018 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315796903999999, 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060220192999999998, 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91018 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.791 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424124+0000", 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0209999999999999e-06 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424135+0000", 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0547000000000001e-05 2024-09-17T22:58:35.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424143+0000", 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484133+0000", 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059990452 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484226+0000", 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2367999999999994e-05 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484336+0000", 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011052099999999999 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91019 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.795 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315762200000002, 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060388084000000002, 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91019 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.796 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:58:35.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424153+0000", 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2819999999999999e-06 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424162+0000", 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424168+0000", 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484367+0000", 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060199257999999999 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484446+0000", 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9029000000000005e-05 2024-09-17T22:58:35.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484539+0000", 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2687000000000002e-05 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91020 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315738489000001, 2024-09-17T22:58:35.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060513375000000001, 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91020 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424175+0000", 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424177+0000", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.978e-06 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424183+0000", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1109999999999996e-06 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424190+0000", 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2300000000000002e-06 2024-09-17T22:58:35.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484556+0000", 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060365758999999998 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484614+0000", 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8233999999999998e-05 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484688+0000", 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3668e-05 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91021 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315713843000001, 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060706860000000001, 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91021 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424201+0000", 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.939e-06 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424207+0000", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424217+0000", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2e-06 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484719+0000", 2024-09-17T22:58:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060502242999999997 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484795+0000", 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6462000000000001e-05 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484906+0000", 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011060700000000001 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91022 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315689590000002, 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060885440999999998, 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91022 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424224+0000", 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424226+0000", 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424232+0000", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1349999999999997e-06 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424240+0000", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4320000000000002e-06 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484935+0000", 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060695168000000001 2024-09-17T22:58:35.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485019+0000", 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4040000000000002e-05 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485109+0000", 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9936000000000001e-05 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315666192999998, 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:35.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:35.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315645416999999, 2024-09-17T22:58:35.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:35.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:35.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315626627, 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:35.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315581416000001, 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:35.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.315547588000001, 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:35.833 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:35.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:35.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:35.838 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:35.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:35.840 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:35.836+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:35.941 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:35.938+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:35.985 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:35.985 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:35.985 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:35.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:36.042 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:36.039+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:36.055 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:36.128 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone49 from 0xce72df5 (overlap 0x5eb99d) to 0x4bc7b0 2024-09-17T22:58:36.143 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:36.140+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:36.146 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:36.147 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:36.156 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:36.199 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-17T22:58:36.244 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:36.241+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:36.304 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:36.304 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:36.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:36.304 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:36.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047812610000001, 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.305 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:36.309 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047796605000002, 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:36.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:36.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047763834999998, 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.314 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.315 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047736096999998, 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:36.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.320 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:36.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047713678000001, 2024-09-17T22:58:36.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:36.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.325 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.328 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047691397999998, 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.329 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:36.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.047483993, 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.334 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:36.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:36.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.046937378000003, 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:36.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:36.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.045866181999997, 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:36.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:36.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.045854726999998, 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.348 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:36.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.352 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.045843750000003, 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:36.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.043769617999999, 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.358 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:36.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.043757304000003, 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:36.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.367 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.043744066999999, 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.368 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.043733238000002, 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:36.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.041774363000002, 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.378 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:36.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:36.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:36.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.041750004999997, 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:36.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.041720787000003, 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:36.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.040954219, 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:36.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:36.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.040941117000003, 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:36.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:36.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.403 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.403 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.403 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.403 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.403 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:36.404 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:36.405 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:36.401+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:36.447 INFO:teuthology.orchestra.run.smithi159.stdout:5838 write 0x200ce92 thru 0x20135c9 (0x6738 bytes) 2024-09-17T22:58:36.505 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:36.502+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:36.540 INFO:teuthology.orchestra.run.smithi159.stdout:5839 read 0x24a8b22 thru 0x24b50f6 (0xc5d5 bytes) 2024-09-17T22:58:36.542 INFO:teuthology.orchestra.run.smithi159.stdout:5841 writesame 0x4a376d7 thru 0x4a44414 data_size 0x7e(0xcd3e bytes) 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144070 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581560+0000", 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0077424959999999998, 2024-09-17T22:58:36.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0077726699999999998, 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144070 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581560+0000", 2024-09-17T22:58:36.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581560+0000", 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581561+0000", 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581562+0000", 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9319999999999999e-06 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581569+0000", 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.443e-06 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581574+0000", 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9420000000000003e-06 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144560 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588623+0000", 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00067981599999999997, 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077685899999999997, 2024-09-17T22:58:36.600 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144560 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588623+0000", 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588623+0000", 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588623+0000", 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588625+0000", 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7239999999999999e-06 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588629+0000", 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588631+0000", 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8789999999999999e-06 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.603 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144071 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581580+0000", 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0077221570000000003, 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0078777269999999993, 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.604 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144071 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581580+0000", 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581580+0000", 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581581+0000", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581583+0000", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.871e-06 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581589+0000", 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0929999999999999e-06 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581594+0000", 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3630000000000002e-06 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144561 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588635+0000", 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00066721599999999999, 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088053299999999997, 2024-09-17T22:58:36.607 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144561 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588635+0000", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588635+0000", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588636+0000", 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588637+0000", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588647+0000", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0054000000000001e-05 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588650+0000", 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9639999999999999e-06 2024-09-17T22:58:36.610 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144072 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581601+0000", 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0077015319999999996, 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0079714759999999999, 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.611 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144072 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581601+0000", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581601+0000", 2024-09-17T22:58:36.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581601+0000", 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581603+0000", 2024-09-17T22:58:36.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.962e-06 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581610+0000", 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3319999999999997e-06 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581615+0000", 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1370000000000001e-06 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.614 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144562 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588655+0000", 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000648067, 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097414099999999998, 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144562 2024-09-17T22:58:36.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588655+0000", 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588655+0000", 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588655+0000", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588656+0000", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4389999999999999e-06 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588660+0000", 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3680000000000002e-06 2024-09-17T22:58:36.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588662+0000", 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5670000000000002e-06 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144073 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581621+0000", 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0076811479999999996, 2024-09-17T22:58:36.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0080627040000000004, 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144073 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581621+0000", 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581621+0000", 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581622+0000", 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581624+0000", 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1469999999999999e-06 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581631+0000", 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7209999999999997e-06 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581636+0000", 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0130000000000001e-06 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144563 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588665+0000", 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00063741999999999998, 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010780049999999999, 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144563 2024-09-17T22:58:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588665+0000", 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588665+0000", 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588665+0000", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588667+0000", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.189e-06 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588670+0000", 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0510000000000001e-06 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588678+0000", 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7970000000000006e-06 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144074 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581642+0000", 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.007660427, 2024-09-17T22:58:36.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0081585779999999997, 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144074 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581642+0000", 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581642+0000", 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581642+0000", 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1e-07 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581644+0000", 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.784e-06 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581651+0000", 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8419999999999999e-06 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581656+0000", 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1710000000000001e-06 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144564 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588686+0000", 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00061675599999999999, 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011616759999999999, 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144564 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588686+0000", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588686+0000", 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588686+0000", 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5899999999999998e-07 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588688+0000", 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.823e-06 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588692+0000", 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.766e-06 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588695+0000", 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.858e-06 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144075 3.16 3.beda3896 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.581663+0000", 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00764002, 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0082244180000000007, 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144075 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.633 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581663+0000", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581663+0000", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581663+0000", 2024-09-17T22:58:36.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581665+0000", 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1239999999999999e-06 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581671+0000", 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8529999999999997e-06 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.581676+0000", 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.198e-06 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144565 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588698+0000", 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000604658, 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012210649999999999, 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.636 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144565 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588698+0000", 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588698+0000", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588698+0000", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588699+0000", 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1599999999999999e-06 2024-09-17T22:58:36.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588702+0000", 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9510000000000002e-06 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588719+0000", 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7149000000000001e-05 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144309 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584087+0000", 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.005215562, 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.005866975, 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144309 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.640 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584087+0000", 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584087+0000", 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584088+0000", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8000000000000004e-07 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584093+0000", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584103+0000", 2024-09-17T22:58:36.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0147e-05 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584111+0000", 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2490000000000004e-06 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144566 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588726+0000", 2024-09-17T22:58:36.643 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00057622900000000004, 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012578310000000001, 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144566 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588726+0000", 2024-09-17T22:58:36.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588726+0000", 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588727+0000", 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2100000000000002e-07 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588729+0000", 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588735+0000", 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8089999999999998e-06 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588739+0000", 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5340000000000001e-06 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.646 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144311 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584124+0000", 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051788720000000002, 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0058945270000000001, 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.647 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144311 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584124+0000", 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584124+0000", 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584124+0000", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1900000000000003e-07 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584127+0000", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584137+0000", 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8870000000000004e-06 2024-09-17T22:58:36.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584143+0000", 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3119999999999999e-06 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144567 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588745+0000", 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00055762900000000002, 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001325358, 2024-09-17T22:58:36.650 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144567 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588745+0000", 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588745+0000", 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588745+0000", 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7e-07 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.652 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588747+0000", 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8840000000000001e-06 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588755+0000", 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6639999999999998e-06 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588759+0000", 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0550000000000001e-06 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.653 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144312 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584156+0000", 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0051466749999999999, 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0059689189999999996, 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.654 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144312 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584156+0000", 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584156+0000", 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584156+0000", 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2e-07 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584159+0000", 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4710000000000001e-06 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.656 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584166+0000", 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584171+0000", 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2100000000000001e-06 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144568 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.657 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588763+0000", 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00053938, 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001420035, 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144568 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.658 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588763+0000", 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588763+0000", 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588763+0000", 2024-09-17T22:58:36.659 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.48e-07 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588765+0000", 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3030000000000001e-06 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588768+0000", 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.083e-06 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.660 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588770+0000", 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4930000000000001e-06 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144313 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584179+0000", 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.005123847, 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006058562, 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.661 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144313 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584179+0000", 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.662 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584179+0000", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584179+0000", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584182+0000", 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2419999999999999e-06 2024-09-17T22:58:36.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584188+0000", 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5250000000000002e-06 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584197+0000", 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1500000000000005e-06 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144569 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588773+0000", 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00052932100000000002, 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001516248, 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144569 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.665 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588773+0000", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588773+0000", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588774+0000", 2024-09-17T22:58:36.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588775+0000", 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588778+0000", 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0139999999999999e-06 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588783+0000", 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.604e-06 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144314 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584206+0000", 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0050970549999999996, 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0061354069999999998, 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.668 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144314 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584206+0000", 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584206+0000", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584206+0000", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2099999999999997e-07 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584209+0000", 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4760000000000001e-06 2024-09-17T22:58:36.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584215+0000", 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7179999999999999e-06 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584220+0000", 2024-09-17T22:58:36.671 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.242e-06 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144570 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588801+0000", 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00050179300000000005, 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001594713, 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.672 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144570 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588801+0000", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588801+0000", 2024-09-17T22:58:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588801+0000", 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588803+0000", 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.44e-06 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588808+0000", 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.147e-06 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588814+0000", 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2160000000000003e-06 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144315 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.675 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584237+0000", 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0050653440000000003, 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062132050000000003, 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144315 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584237+0000", 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584237+0000", 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584238+0000", 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584240+0000", 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5229999999999998e-06 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584248+0000", 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4529999999999996e-06 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584253+0000", 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.322e-06 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144571 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588821+0000", 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00048206099999999999, 2024-09-17T22:58:36.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001684341, 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144571 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.680 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588821+0000", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588821+0000", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588821+0000", 2024-09-17T22:58:36.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6400000000000003e-07 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588823+0000", 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588831+0000", 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2409999999999995e-06 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588837+0000", 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.789e-06 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144316 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584260+0000", 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0050425799999999996, 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0062930379999999999, 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.683 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144316 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584260+0000", 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584260+0000", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584261+0000", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584263+0000", 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1210000000000001e-06 2024-09-17T22:58:36.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584269+0000", 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0730000000000001e-06 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584274+0000", 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8169999999999998e-06 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144572 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588845+0000", 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00045778000000000003, 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017363669999999999, 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144572 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.687 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588845+0000", 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588845+0000", 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588845+0000", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.27e-07 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588847+0000", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1009999999999999e-06 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588854+0000", 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6420000000000001e-06 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588860+0000", 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3230000000000003e-06 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144317 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584280+0000", 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0050225469999999996, 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063283159999999996, 2024-09-17T22:58:36.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144317 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584280+0000", 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584280+0000", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584280+0000", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584283+0000", 2024-09-17T22:58:36.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.187e-06 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584289+0000", 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0800000000000002e-06 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584297+0000", 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9500000000000001e-06 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144573 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588870+0000", 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00043246400000000003, 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017651209999999999, 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144573 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.694 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588870+0000", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588870+0000", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588871+0000", 2024-09-17T22:58:36.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588873+0000", 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588879+0000", 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5520000000000001e-06 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588885+0000", 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3469999999999998e-06 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144318 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584303+0000", 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049991649999999999, 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0063587019999999999, 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.697 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144318 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584303+0000", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584303+0000", 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584304+0000", 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584306+0000", 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1090000000000001e-06 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584312+0000", 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2709999999999998e-06 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584320+0000", 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.221e-06 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.700 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144574 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588891+0000", 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00041111400000000002, 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001797603, 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144574 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588891+0000", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588891+0000", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588892+0000", 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588894+0000", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9539999999999998e-06 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588902+0000", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588909+0000", 2024-09-17T22:58:36.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3220000000000004e-06 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144319 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584328+0000", 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049749260000000002, 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0064122830000000004, 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.704 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144319 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584328+0000", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584328+0000", 2024-09-17T22:58:36.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584328+0000", 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584330+0000", 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584336+0000", 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1380000000000004e-06 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584341+0000", 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7230000000000004e-06 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144575 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.707 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588917+0000", 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00038521999999999997, 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018783509999999999, 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144575 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588917+0000", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588917+0000", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588918+0000", 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588924+0000", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9440000000000002e-06 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588931+0000", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.939e-06 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588936+0000", 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.383e-06 2024-09-17T22:58:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144320 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584347+0000", 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049556979999999997, 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006503773, 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.711 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144320 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584347+0000", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584347+0000", 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584347+0000", 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584349+0000", 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9580000000000001e-06 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584355+0000", 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8030000000000002e-06 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584360+0000", 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5569999999999997e-06 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144576 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588943+0000", 2024-09-17T22:58:36.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00035994199999999998, 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019628010000000001, 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144576 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588943+0000", 2024-09-17T22:58:36.715 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588943+0000", 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588943+0000", 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588945+0000", 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588951+0000", 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0669999999999997e-06 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588957+0000", 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.93e-06 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144321 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584365+0000", 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049378119999999998, 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.006618596, 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144321 2024-09-17T22:58:36.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584365+0000", 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584365+0000", 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584365+0000", 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584367+0000", 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0810000000000001e-06 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584373+0000", 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8660000000000002e-06 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584378+0000", 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4449999999999998e-06 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144577 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588963+0000", 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00033911100000000001, 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020732300000000001, 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.721 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144577 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588963+0000", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588963+0000", 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588964+0000", 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588966+0000", 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5059999999999998e-06 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588973+0000", 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2339999999999999e-06 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588978+0000", 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.921e-06 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.724 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144322 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584383+0000", 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049196099999999996, 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0067111460000000003, 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144322 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584383+0000", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584383+0000", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584383+0000", 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:36.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584385+0000", 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584391+0000", 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584395+0000", 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3719999999999998e-06 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144578 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588984+0000", 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00031853099999999999, 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00216467, 2024-09-17T22:58:36.728 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144578 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588984+0000", 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588984+0000", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588984+0000", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588986+0000", 2024-09-17T22:58:36.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9609999999999999e-06 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588993+0000", 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588998+0000", 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1150000000000002e-06 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144323 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584401+0000", 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0049015509999999997, 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068010259999999999, 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144323 2024-09-17T22:58:36.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584401+0000", 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584401+0000", 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584401+0000", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.89e-07 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584408+0000", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4500000000000001e-06 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584414+0000", 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3690000000000003e-06 2024-09-17T22:58:36.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584420+0000", 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3850000000000001e-06 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144579 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589007+0000", 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00029563000000000001, 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022341840000000002, 2024-09-17T22:58:36.735 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144579 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589007+0000", 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589007+0000", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589008+0000", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589010+0000", 2024-09-17T22:58:36.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0059999999999999e-06 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589016+0000", 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.764e-06 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589022+0000", 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3109999999999996e-06 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144324 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584436+0000", 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.004866988, 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068448099999999998, 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144324 2024-09-17T22:58:36.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584436+0000", 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584436+0000", 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584436+0000", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9599999999999999e-07 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584439+0000", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.351e-06 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584446+0000", 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3180000000000001e-06 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584451+0000", 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1660000000000002e-06 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144580 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589028+0000", 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000274331, 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022825340000000001, 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.742 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144580 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589028+0000", 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589028+0000", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589029+0000", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589031+0000", 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.95e-06 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589037+0000", 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9730000000000002e-06 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589042+0000", 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1089999999999997e-06 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144325 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584458+0000", 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0048446670000000004, 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0068801699999999997, 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144325 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584458+0000", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584458+0000", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584458+0000", 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6899999999999998e-07 2024-09-17T22:58:36.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584464+0000", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5799999999999999e-06 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584470+0000", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3169999999999998e-06 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584475+0000", 2024-09-17T22:58:36.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9320000000000004e-06 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144581 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589048+0000", 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000254493, 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002318358, 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144581 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589048+0000", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589048+0000", 2024-09-17T22:58:36.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589048+0000", 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589054+0000", 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5040000000000002e-06 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589061+0000", 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5060000000000001e-06 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589066+0000", 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144326 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584483+0000", 2024-09-17T22:58:36.752 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0048196619999999997, 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069236489999999996, 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144326 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584483+0000", 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584483+0000", 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584483+0000", 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6899999999999998e-07 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584486+0000", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584492+0000", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4609999999999996e-06 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584497+0000", 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9010000000000002e-06 2024-09-17T22:58:36.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144582 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589072+0000", 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00023029499999999999, 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023767250000000001, 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.756 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144582 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589072+0000", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589072+0000", 2024-09-17T22:58:36.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589073+0000", 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0400000000000002e-07 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589075+0000", 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589081+0000", 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3749999999999999e-06 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589086+0000", 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9989999999999999e-06 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144327 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584503+0000", 2024-09-17T22:58:36.759 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0047991750000000001, 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0069737970000000003, 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144327 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584503+0000", 2024-09-17T22:58:36.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584503+0000", 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584504+0000", 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584506+0000", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0779999999999998e-06 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584513+0000", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0859999999999996e-06 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584518+0000", 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8919999999999999e-06 2024-09-17T22:58:36.762 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144583 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589092+0000", 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00021024699999999999, 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024139220000000002, 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.763 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144583 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589092+0000", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589092+0000", 2024-09-17T22:58:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589093+0000", 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589095+0000", 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589101+0000", 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589105+0000", 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6410000000000001e-06 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144328 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584525+0000", 2024-09-17T22:58:36.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0047773690000000001, 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00700835, 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144328 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584525+0000", 2024-09-17T22:58:36.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584525+0000", 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584526+0000", 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.08e-07 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584528+0000", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1859999999999999e-06 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584535+0000", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9619999999999996e-06 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584540+0000", 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144584 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589112+0000", 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000190767, 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024554949999999998, 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144584 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589112+0000", 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589112+0000", 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589112+0000", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.03e-07 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589114+0000", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-06 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589121+0000", 2024-09-17T22:58:36.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.241e-06 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589126+0000", 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1290000000000004e-06 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144329 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584557+0000", 2024-09-17T22:58:36.773 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.004745979, 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0070626090000000001, 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144329 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584557+0000", 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584557+0000", 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584557+0000", 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9599999999999999e-07 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584560+0000", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3989999999999998e-06 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584568+0000", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4209999999999995e-06 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584574+0000", 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6679999999999998e-06 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.776 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144585 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589132+0000", 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000170889, 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025437659999999998, 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.777 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144585 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589132+0000", 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589132+0000", 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589132+0000", 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4999999999999998e-07 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589134+0000", 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9800000000000001e-06 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.779 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589140+0000", 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6720000000000001e-06 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589145+0000", 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8110000000000002e-06 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.780 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144330 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584580+0000", 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0047221800000000003, 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0071530480000000004, 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144330 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584580+0000", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584580+0000", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584581+0000", 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:58:36.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584583+0000", 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9439999999999999e-06 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584590+0000", 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2060000000000001e-06 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584595+0000", 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2700000000000004e-06 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144586 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589151+0000", 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00015154900000000001, 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026382630000000001, 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144586 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589151+0000", 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589151+0000", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589151+0000", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589154+0000", 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1380000000000001e-06 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589160+0000", 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2979999999999997e-06 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589165+0000", 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9649999999999999e-06 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.787 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144331 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584601+0000", 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0047018429999999998, 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.007237991, 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144331 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.788 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584601+0000", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584601+0000", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584601+0000", 2024-09-17T22:58:36.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584603+0000", 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8419999999999999e-06 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584609+0000", 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8220000000000003e-06 2024-09-17T22:58:36.790 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584613+0000", 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5990000000000004e-06 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144587 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589171+0000", 2024-09-17T22:58:36.791 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.000131633, 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0027216340000000001, 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144587 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589171+0000", 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589171+0000", 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589172+0000", 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3499999999999996e-07 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589178+0000", 2024-09-17T22:58:36.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589184+0000", 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7129999999999999e-06 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589189+0000", 2024-09-17T22:58:36.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9830000000000004e-06 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144332 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584619+0000", 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0046838569999999996, 2024-09-17T22:58:36.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00733273, 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144332 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584619+0000", 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584619+0000", 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584619+0000", 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2099999999999998e-07 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584621+0000", 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7409999999999999e-06 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584627+0000", 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584634+0000", 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2989999999999999e-06 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144588 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589196+0000", 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00010616, 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028098450000000001, 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144588 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589196+0000", 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589196+0000", 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589197+0000", 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589199+0000", 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:58:36.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589205+0000", 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4880000000000004e-06 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589211+0000", 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1499999999999998e-06 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144333 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.584641+0000", 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.0046615370000000003, 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0074225280000000003, 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144333 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584641+0000", 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584641+0000", 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584641+0000", 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584644+0000", 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.221e-06 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584652+0000", 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6319999999999997e-06 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.584659+0000", 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3330000000000002e-06 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144589 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.589217+0000", 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.5679000000000004e-05, 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0029018529999999998, 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144589 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589217+0000", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589217+0000", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589217+0000", 2024-09-17T22:58:36.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589219+0000", 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1210000000000001e-06 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589225+0000", 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9519999999999999e-06 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.589230+0000", 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.977e-06 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144558 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588558+0000", 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00074483199999999996, 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036209430000000002, 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.812 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144558 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588558+0000", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588558+0000", 2024-09-17T22:58:36.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588559+0000", 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8299999999999996e-07 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588566+0000", 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0809999999999996e-06 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588577+0000", 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1181e-05 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588582+0000", 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.383e-06 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:144559 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-17T22:58:36.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:36.588592+0000", 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00071062399999999998, 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0036415430000000001, 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 144559 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:36.816 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588592+0000", 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588592+0000", 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588592+0000", 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1699999999999999e-07 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588594+0000", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.728e-06 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588612+0000", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8306000000000001e-05 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:36.588617+0000", 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2050000000000004e-06 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.818 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:36.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.819 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:36.819 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T22:58:36.819 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 62 2024-09-17T22:58:36.819 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:36.819 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:36.821 INFO:teuthology.orchestra.run.smithi159.stdout:5842 read 0x8559391 thru 0x855fa35 (0x66a5 bytes) 2024-09-17T22:58:36.821 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:36.822 INFO:teuthology.orchestra.run.smithi159.stdout:5844 trunc from 0xce72df5 to 0x959e68f 2024-09-17T22:58:36.822 INFO:teuthology.orchestra.run.smithi159.stdout:5846 write 0x9644d20 thru 0x964cc7f (0x7f60 bytes) 2024-09-17T22:58:36.822 INFO:teuthology.orchestra.run.smithi159.stdout:5847 punch from 0x3b72abe to 0x3b7fccb, (0xd20d bytes) 2024-09-17T22:58:36.822 INFO:teuthology.orchestra.run.smithi159.stdout:5850 clone 51 order 18 su 65536 sc 6 2024-09-17T22:58:36.822 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:36.823 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:36.817+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:36.921 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:36.919+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:36.941 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:36.941 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:36.957 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:36.958 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:36.958 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:36.966 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:36.966 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:36.966 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:36.966 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:36.966 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:36.974 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:36.974 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:36.975 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:36.990 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:36.990 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:37.007 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:37.008 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:37.022 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:37.019+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:37.024 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:37.024 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:37.041 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:37.042 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:37.044 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:37.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:37.065 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:58:37.084 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.084 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:37.131 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone50 from 0x964cc80 (overlap 0x959e68f) to 0x68bbafd 2024-09-17T22:58:37.134 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:37.131+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:37.224 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:37.220+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:37.266 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.266 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.278 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:37.325 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:37.323+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:37.426 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:37.423+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:37.461 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.461 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.472 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:37.556 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:37.553+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:37.626 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.627 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.627 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:37.624+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:37.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:37.699 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-17T22:58:37.720 INFO:teuthology.orchestra.run.smithi159.stdout:5852 write 0xd1b5cbb thru 0xd1b8818 (0x2b5e bytes) 2024-09-17T22:58:37.723 INFO:teuthology.orchestra.run.smithi159.stdout:5853 read 0x58fe01b thru 0x5901707 (0x36ed bytes) 2024-09-17T22:58:37.723 INFO:teuthology.orchestra.run.smithi159.stdout:5854 read 0x6423acf thru 0x6427e88 (0x43ba bytes) 2024-09-17T22:58:37.725 INFO:teuthology.orchestra.run.smithi159.stdout:5856 punch from 0x1f981c to 0x1fd195, (0x3979 bytes) 2024-09-17T22:58:37.725 INFO:teuthology.orchestra.run.smithi159.stdout:5859 write 0x7b1233b thru 0x7b1a661 (0x8327 bytes) 2024-09-17T22:58:37.728 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:37.726+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:37.741 INFO:teuthology.orchestra.run.smithi159.stdout:5860 trunc from 0xd1b8819 to 0x595ced1 2024-09-17T22:58:37.753 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.753 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.766 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:37.786 INFO:teuthology.orchestra.run.smithi159.stdout:5861 write 0xd6590cf thru 0xd6660c9 (0xcffb bytes) 2024-09-17T22:58:37.789 INFO:teuthology.orchestra.run.smithi159.stdout:5862 writesame 0x18ead03 thru 0x18f8860 data_size 0x2b(0xdb5e bytes) 2024-09-17T22:58:37.804 INFO:teuthology.orchestra.run.smithi159.stdout:5863 writesame 0x861baad thru 0x861fe9b data_size 0x33(0x43ef bytes) 2024-09-17T22:58:37.808 INFO:teuthology.orchestra.run.smithi159.stdout:5864 read 0xbb515e4 thru 0xbb5220c (0xc29 bytes) 2024-09-17T22:58:37.808 INFO:teuthology.orchestra.run.smithi159.stdout:5866 read 0xaadba7f thru 0xaae70fb (0xb67d bytes) 2024-09-17T22:58:37.808 INFO:teuthology.orchestra.run.smithi159.stdout:5867 writesame 0x58a86ca thru 0x58aedec data_size 0xd(0x6723 bytes) 2024-09-17T22:58:37.809 INFO:teuthology.orchestra.run.smithi159.stdout:5871 write 0x2b01152 thru 0x2b0723e (0x60ed bytes) 2024-09-17T22:58:37.819 INFO:teuthology.orchestra.run.smithi159.stdout:5872 write 0x2b4e5d6 thru 0x2b53d1e (0x5749 bytes) 2024-09-17T22:58:37.829 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:37.826+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:37.832 INFO:teuthology.orchestra.run.smithi159.stdout:5874 read 0x52719d5 thru 0x5274a24 (0x3050 bytes) 2024-09-17T22:58:37.833 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.833 INFO:teuthology.orchestra.run.smithi159.stdout:5877 punch from 0x30cf801 to 0x30db25c, (0xba5b bytes) 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:5878 punch from 0x48ec62a to 0x48fbb46, (0xf51c bytes) 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:5883 read 0x5b1e836 thru 0x5b25bcd (0x7398 bytes) 2024-09-17T22:58:37.834 INFO:teuthology.orchestra.run.smithi159.stdout:5884 write 0x1f55d77 thru 0x1f649a8 (0xec32 bytes) 2024-09-17T22:58:37.842 INFO:teuthology.orchestra.run.smithi159.stdout:5885 writesame 0x23323c8 thru 0x23375ad data_size 0xb(0x51e6 bytes) 2024-09-17T22:58:37.848 INFO:teuthology.orchestra.run.smithi159.stdout:5886 writesame 0x958e78a thru 0x959c866 data_size 0x1d(0xe0dd bytes) 2024-09-17T22:58:37.852 INFO:teuthology.orchestra.run.smithi159.stdout:5888 read 0x31818d1 thru 0x318721e (0x594e bytes) 2024-09-17T22:58:37.854 INFO:teuthology.orchestra.run.smithi159.stdout:5889 read 0x9de0dc8 thru 0x9deffb4 (0xf1ed bytes) 2024-09-17T22:58:37.854 INFO:teuthology.orchestra.run.smithi159.stdout:5890 writesame 0x95ed74e thru 0x95f4e94 data_size 0x1f(0x7747 bytes) 2024-09-17T22:58:37.856 INFO:teuthology.orchestra.run.smithi159.stdout:5891 writesame 0x74b4292 thru 0x74bef22 data_size 0x1(0xac91 bytes) 2024-09-17T22:58:37.877 INFO:teuthology.orchestra.run.smithi159.stdout:5892 writesame 0x460c4dc thru 0x461adc1 data_size 0x39(0xe8e6 bytes) 2024-09-17T22:58:37.893 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.893 INFO:teuthology.orchestra.run.smithi159.stdout:5894 write 0xaa5f3ed thru 0xaa6ab1c (0xb730 bytes) 2024-09-17T22:58:37.896 INFO:teuthology.orchestra.run.smithi159.stdout:5895 read 0x3428bee thru 0x3431044 (0x8457 bytes) 2024-09-17T22:58:37.898 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:37.898 INFO:teuthology.orchestra.run.smithi159.stdout:5899 read 0xc02d7b0 thru 0xc038ecd (0xb71e bytes) 2024-09-17T22:58:37.907 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:37.907 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:37.912 INFO:teuthology.orchestra.run.smithi159.stdout:5900 trunc from 0xd6660ca to 0x1ea078f 2024-09-17T22:58:37.921 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:37.930 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:37.927+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:37.947 INFO:teuthology.orchestra.run.smithi159.stdout:5902 write 0x7230ef2 thru 0x7238324 (0x7433 bytes) 2024-09-17T22:58:37.956 INFO:teuthology.orchestra.run.smithi159.stdout:5904 write 0x7f0af72 thru 0x7f0c4f2 (0x1581 bytes) 2024-09-17T22:58:37.959 INFO:teuthology.orchestra.run.smithi159.stdout:5905 punch from 0x20b6ce6 to 0x20c1433, (0xa74d bytes) 2024-09-17T22:58:37.959 INFO:teuthology.orchestra.run.smithi159.stdout:5908 read 0x2e886ba thru 0x2e90502 (0x7e49 bytes) 2024-09-17T22:58:37.959 INFO:teuthology.orchestra.run.smithi159.stdout:5909 write 0x623f79e thru 0x624a59e (0xae01 bytes) 2024-09-17T22:58:37.962 INFO:teuthology.orchestra.run.smithi159.stdout:5911 clone 52 order 21 su 32768 sc 8 2024-09-17T22:58:38.031 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:38.028+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:38.051 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:38.051 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:38.059 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:38.138 INFO:teuthology.orchestra.run.smithi159.stdout:truncating image image_client.0-clone51 from 0x7f0c4f3 (overlap 0x1ea078f) to 0xf503c7 2024-09-17T22:58:38.138 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:38.135+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:38.232 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:38.229+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:38.257 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:38.258 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:38.271 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:38.333 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:38.330+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:38.412 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-17T22:58:38.425 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:38.425 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:38.434 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:38.432+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:38.437 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:38.537 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:38.533+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:38.624 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:38.624 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:38.637 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:38.633+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:38.639 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:38.738 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:38.734+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:38.838 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:38.835+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:38.865 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:38.865 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:38.878 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:38.890 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:58:38.890 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:38.890 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T22:58:38.890 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:58:38.890 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:58:38.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T22:58:38.939 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:38.936+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:39.040 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:39.037+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:39.046 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:39.049 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:39.049 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:39.065 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:39.080 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:58:39.102 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:39.102 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:58:39.102 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:39.141 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:39.138+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:39.242 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:39.239+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:39.292 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:39.292 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:39.304 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:39.348 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:39.345+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:39.368 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:39.368 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:39.402 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:39.403 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:39.403 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:39.424 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:39.424 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:39.433 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:39.438 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:39.438 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:39.444 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:39.441+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:39.469 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:39.470 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:39.500 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:39.500 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:39.530 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:39.530 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:39.544 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:39.541+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:39.559 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:39.560 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:39.563 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:39.579 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:39.579 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:39.579 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:39.588 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:39.674 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:39.671+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.735 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89486 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.346731149, 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059383219000000001, 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89486 2024-09-17T22:58:39.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385238+0000", 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385240+0000", 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9190000000000002e-06 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.738 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385245+0000", 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.639e-06 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385248+0000", 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1319999999999998e-06 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444094+0000", 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058845916999999998 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444207+0000", 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000113261 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444621+0000", 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041399299999999998 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.740 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89487 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.346716719, 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059624093000000003, 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89487 2024-09-17T22:58:39.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385253+0000", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.15e-07 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385254+0000", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2389999999999999e-06 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385257+0000", 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3450000000000002e-06 2024-09-17T22:58:39.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385260+0000", 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.785e-06 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444667+0000", 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059407142000000003 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444750+0000", 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2232999999999997e-05 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444877+0000", 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012703399999999999 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89488 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 257024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:39.745 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.346705914000001, 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059823605000000002, 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89488 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:39.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385263+0000", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385264+0000", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.22e-07 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385265+0000", 2024-09-17T22:58:39.747 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.206e-06 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385268+0000", 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1839999999999999e-06 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385271+0000", 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7389999999999999e-06 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.748 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444898+0000", 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059626894999999999 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.444967+0000", 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9573999999999995e-05 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445087+0000", 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011978500000000001 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.749 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.346695581999999, 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.750 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:39.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.754 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91012 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 220160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.308001444999999, 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059095525000000003, 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91012 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423968+0000", 2024-09-17T22:58:39.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423970+0000", 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423982+0000", 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:58:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423988+0000", 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482905+0000", 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.058917737999999997 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.482969+0000", 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4054999999999996e-05 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483063+0000", 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3968999999999994e-05 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91013 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307971258999999, 2024-09-17T22:58:39.759 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059240328000000002, 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91013 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.423998+0000", 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.761 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424000+0000", 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0760000000000001e-06 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424012+0000", 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1525000000000001e-05 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424018+0000", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6779999999999997e-06 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483090+0000", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059072066999999999 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483131+0000", 2024-09-17T22:58:39.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1427000000000002e-05 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483238+0000", 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000107161 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.764 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91014 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307944576000001, 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059357291999999999, 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91014 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424025+0000", 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:58:39.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424027+0000", 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424035+0000", 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6389999999999992e-06 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424040+0000", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9389999999999996e-06 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483266+0000", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059226144000000001 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483326+0000", 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9531999999999999e-05 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483382+0000", 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6484999999999999e-05 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91015 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307924108999998, 2024-09-17T22:58:39.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059570633999999997, 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91015 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.770 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424045+0000", 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:39.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424047+0000", 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424050+0000", 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5640000000000001e-06 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424053+0000", 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6070000000000002e-06 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483410+0000", 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059356475999999998 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483489+0000", 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9263000000000003e-05 2024-09-17T22:58:39.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483616+0000", 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012700299999999999 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91016 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 231424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307912006999999, 2024-09-17T22:58:39.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059795158000000001, 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91016 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:39.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424057+0000", 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424058+0000", 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0699999999999997e-07 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424060+0000", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8360000000000001e-06 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424068+0000", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424072+0000", 2024-09-17T22:58:39.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8249999999999998e-06 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483668+0000", 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059595608000000001 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483751+0000", 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3778999999999999e-05 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.778 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483852+0000", 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000101059 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91017 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307878645999999, 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059982901999999998, 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.779 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91017 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.780 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424091+0000", 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.96e-07 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.781 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424096+0000", 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0710000000000002e-06 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424103+0000", 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8730000000000001e-06 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424108+0000", 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:58:39.782 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483886+0000", 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059777978000000002 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.483971+0000", 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5172000000000004e-05 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.783 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484074+0000", 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000102697 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91018 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307853341000001, 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060220192999999998, 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.784 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91018 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.785 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424116+0000", 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.01e-07 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.786 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424124+0000", 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0209999999999999e-06 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424135+0000", 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0547000000000001e-05 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.787 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424143+0000", 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8830000000000006e-06 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484133+0000", 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059990452 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.788 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484226+0000", 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2367999999999994e-05 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484336+0000", 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011052099999999999 2024-09-17T22:58:39.789 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.790 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91019 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307818637, 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060388084000000002, 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.791 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91019 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424151+0000", 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424153+0000", 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2819999999999999e-06 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424162+0000", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0089999999999996e-06 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424168+0000", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484367+0000", 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060199257999999999 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484446+0000", 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9029000000000005e-05 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484539+0000", 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2687000000000002e-05 2024-09-17T22:58:39.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.796 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.796 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91020 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307794926, 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060513375000000001, 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.797 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91020 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.798 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424174+0000", 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424175+0000", 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424177+0000", 2024-09-17T22:58:39.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.978e-06 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424183+0000", 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1109999999999996e-06 2024-09-17T22:58:39.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424190+0000", 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2300000000000002e-06 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484556+0000", 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060365758999999998 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484614+0000", 2024-09-17T22:58:39.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8233999999999998e-05 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484688+0000", 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3668e-05 2024-09-17T22:58:39.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91021 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:39.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.30777028, 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060706860000000001, 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91021 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:39.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424199+0000", 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424201+0000", 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.939e-06 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.810 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424207+0000", 2024-09-17T22:58:39.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:58:39.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424217+0000", 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2e-06 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484719+0000", 2024-09-17T22:58:39.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060502242999999997 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484795+0000", 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6462000000000001e-05 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484906+0000", 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00011060700000000001 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.814 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91022 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:39.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307746027, 2024-09-17T22:58:39.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060885440999999998, 2024-09-17T22:58:39.815 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91022 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:39.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424223+0000", 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424224+0000", 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424226+0000", 2024-09-17T22:58:39.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424232+0000", 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1349999999999997e-06 2024-09-17T22:58:39.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424240+0000", 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4320000000000002e-06 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.484935+0000", 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060695168000000001 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485019+0000", 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4040000000000002e-05 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485109+0000", 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9936000000000001e-05 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.824 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:39.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307722630000001, 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:39.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:39.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:39.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:39.832 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307701854000001, 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.833 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:39.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:39.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:39.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:39.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.842 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307683063999999, 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.843 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:39.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:39.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:39.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:39.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:39.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307637852999999, 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.852 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:39.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:39.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.307604025, 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:39.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:39.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.863 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:39.863 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:39.863 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:39.865 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:39.865 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:39.861+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5912 read 0x235fa5e thru 0x2363044 (0x35e7 bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5913 trunc from 0x7f0c4f3 to 0xa916b5b 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5915 writesame 0xb3ff9a1 thru 0xb40ef49 data_size 0x1(0xf5a9 bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5917 trunc from 0xb40ef4a to 0x60b8086 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5919 write 0x73a44c6 thru 0x73a92ea (0x4e25 bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5920 write 0xec1203c thru 0xec1c088 (0xa04d bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5921 writesame 0x2034e34 thru 0x203fdcf data_size 0x2(0xaf9c bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5922 writesame 0x951a6e8 thru 0x951faad data_size 0x1(0x53c6 bytes) 2024-09-17T22:58:39.866 INFO:teuthology.orchestra.run.smithi159.stdout:5923 write 0x8c6ad72 thru 0x8c6ddad (0x303c bytes) 2024-09-17T22:58:39.867 INFO:teuthology.orchestra.run.smithi159.stdout:5924 write 0x37e4f22 thru 0x37f0e33 (0xbf12 bytes) 2024-09-17T22:58:39.867 INFO:teuthology.orchestra.run.smithi159.stdout:5926 write 0x169c2ac thru 0x16a5201 (0x8f56 bytes) 2024-09-17T22:58:39.899 INFO:teuthology.orchestra.run.smithi159.stdout:5927 trunc from 0xec1c089 to 0x50ede04 2024-09-17T22:58:39.919 INFO:teuthology.orchestra.run.smithi159.stdout:5929 punch from 0x30c5cbd to 0x30c8ddd, (0x3120 bytes) 2024-09-17T22:58:39.919 INFO:teuthology.orchestra.run.smithi159.stdout:5930 read 0x4886001 thru 0x48870fb (0x10fb bytes) 2024-09-17T22:58:39.920 INFO:teuthology.orchestra.run.smithi159.stdout:5931 flatten 2024-09-17T22:58:39.965 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:39.962+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:40.042 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.042 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:40.042 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:40.042 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.051 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:40.087 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:58:40.092 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:40.089+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:40.166 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:40.164+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:40.216 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:40.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.226 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:40.264 INFO:teuthology.orchestra.run.smithi159.stdout:5934 punch from 0x37313ce to 0x3737342, (0x5f74 bytes) 2024-09-17T22:58:40.269 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:40.266+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:40.277 INFO:teuthology.orchestra.run.smithi159.stdout:5936 write 0xadd9bef thru 0xade41da (0xa5ec bytes) 2024-09-17T22:58:40.285 INFO:teuthology.orchestra.run.smithi159.stdout:5937 trunc from 0xade41db to 0x6ce05b2 2024-09-17T22:58:40.294 INFO:teuthology.orchestra.run.smithi159.stdout:5938 write 0x4ad88e0 thru 0x4ad976a (0xe8b bytes) 2024-09-17T22:58:40.297 INFO:teuthology.orchestra.run.smithi159.stdout:5939 read 0xc72453 thru 0xc74d14 (0x28c2 bytes) 2024-09-17T22:58:40.297 INFO:teuthology.orchestra.run.smithi159.stdout:5940 read 0x1fe12 thru 0x2519d (0x538c bytes) 2024-09-17T22:58:40.304 INFO:teuthology.orchestra.run.smithi159.stdout:5942 write 0x8a2696e thru 0x8a2763d (0xcd0 bytes) 2024-09-17T22:58:40.307 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:40.307 INFO:teuthology.orchestra.run.smithi159.stdout:5945 write 0x7be8534 thru 0x7bf2fbc (0xaa89 bytes) 2024-09-17T22:58:40.310 INFO:teuthology.orchestra.run.smithi159.stdout:5946 trunc from 0x8a2763e to 0x8fa6f99 2024-09-17T22:58:40.315 INFO:teuthology.orchestra.run.smithi159.stdout:5948 read 0x3aba087 thru 0x3ac43be (0xa338 bytes) 2024-09-17T22:58:40.326 INFO:teuthology.orchestra.run.smithi159.stdout:5949 write 0xddbd59a thru 0xddbea82 (0x14e9 bytes) 2024-09-17T22:58:40.329 INFO:teuthology.orchestra.run.smithi159.stdout:5952 write 0x624de7c thru 0x625dc3d (0xfdc2 bytes) 2024-09-17T22:58:40.333 INFO:teuthology.orchestra.run.smithi159.stdout:5954 read 0x7e5d373 thru 0x7e60629 (0x32b7 bytes) 2024-09-17T22:58:40.333 INFO:teuthology.orchestra.run.smithi159.stdout:5955 write 0x13f1290 thru 0x13fd9a9 (0xc71a bytes) 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.335 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079252586999999, 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:40.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079236581999993, 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.341 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:40.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079203812000003, 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:40.345 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.346 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:40.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079176074000003, 2024-09-17T22:58:40.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:40.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:40.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079153654999999, 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.079131375000003, 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.361 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:40.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:40.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.078923970000005, 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:40.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.078377355000001, 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:40.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.077306159000003, 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:40.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:40.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.077294703999996, 2024-09-17T22:58:40.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:40.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:40.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.077283726999994, 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:40.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.388 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.075209595000004, 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:40.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.075197281000001, 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:40.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:40.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.075184043999997, 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.398 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:40.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.075173215000007, 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:40.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.073214340000007, 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:40.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.073189982000002, 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:40.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.073160763999994, 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:40.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:40.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.072394196000005, 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:40.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:40.425 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.072381093999994, 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:40.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.430 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.430 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.432 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:5956 read 0xc448649 thru 0xc44d6f4 (0x50ac bytes) 2024-09-17T22:58:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:5957 read 0x3781f15 thru 0x378ef8a (0xd076 bytes) 2024-09-17T22:58:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:5958 punch from 0x2911797 to 0x291df13, (0xc77c bytes) 2024-09-17T22:58:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:5959 read 0x6840ca0 thru 0x684d707 (0xca68 bytes) 2024-09-17T22:58:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:5960 read 0x8aeba4 thru 0x8af8c7 (0xd24 bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5962 punch from 0x72b3029 to 0x72bef75, (0xbf4c bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5965 trunc from 0xddbea83 to 0x360c432 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5966 writesame 0xc35db0 thru 0xc39799 data_size 0x3(0x39ea bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5967 trunc from 0x360c432 to 0xdba576a 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5968 read 0x48ae3ec thru 0x48b3f87 (0x5b9c bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5970 punch from 0x3bf90a4 to 0x3bff159, (0x60b5 bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5972 write 0xa51ecb4 thru 0xa529e80 (0xb1cd bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5973 read 0x7d7aa9b thru 0x7d838d5 (0x8e3b bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5974 trunc from 0xdba576a to 0x56263b7 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5975 writesame 0xac8e6b2 thru 0xac92bae data_size 0x1d(0x44fd bytes) 2024-09-17T22:58:40.434 INFO:teuthology.orchestra.run.smithi159.stdout:5977 writesame 0x1049a09 thru 0x1050762 data_size 0x1(0x6d5a bytes) 2024-09-17T22:58:40.435 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:40.429+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:40.435 INFO:teuthology.orchestra.run.smithi159.stdout:5978 read 0x5ca7616 thru 0x5cad478 (0x5e63 bytes) 2024-09-17T22:58:40.435 INFO:teuthology.orchestra.run.smithi159.stdout:compare and write disabled 2024-09-17T22:58:40.435 INFO:teuthology.orchestra.run.smithi159.stdout:5980 read 0x2930b7e thru 0x2940a94 (0xff17 bytes) 2024-09-17T22:58:40.435 INFO:teuthology.orchestra.run.smithi159.stdout:5981 write 0x3d1239b thru 0x3d142dd (0x1f43 bytes) 2024-09-17T22:58:40.436 INFO:teuthology.orchestra.run.smithi159.stdout:5983 read 0x123f1e5 thru 0x12490be (0x9eda bytes) 2024-09-17T22:58:40.438 INFO:teuthology.orchestra.run.smithi159.stdout:5984 writesame 0x299cc47 thru 0x29a860b data_size 0x1(0xb9c5 bytes) 2024-09-17T22:58:40.460 INFO:teuthology.orchestra.run.smithi159.stdout:5985 read 0x7186db3 thru 0x718d801 (0x6a4f bytes) 2024-09-17T22:58:40.460 INFO:teuthology.orchestra.run.smithi159.stdout:5986 punch from 0x4e98b8 to 0x4eb730, (0x1e78 bytes) 2024-09-17T22:58:40.460 INFO:teuthology.orchestra.run.smithi159.stdout:5987 punch from 0x4fd7ecd to 0x4fdc92a, (0x4a5d bytes) 2024-09-17T22:58:40.460 INFO:teuthology.orchestra.run.smithi159.stdout:5988 write 0x6807306 thru 0x681367e (0xc379 bytes) 2024-09-17T22:58:40.464 INFO:teuthology.orchestra.run.smithi159.stdout:5990 read 0x9519f7 thru 0x95bab5 (0xa0bf bytes) 2024-09-17T22:58:40.464 INFO:teuthology.orchestra.run.smithi159.stdout:5992 punch from 0x6d40c30 to 0x6d42c40, (0x2010 bytes) 2024-09-17T22:58:40.464 INFO:teuthology.orchestra.run.smithi159.stdout:5993 read 0x5b22d6e thru 0x5b306fb (0xd98e bytes) 2024-09-17T22:58:40.465 INFO:teuthology.orchestra.run.smithi159.stdout:5994 write 0x24711e3 thru 0x247a500 (0x931e bytes) 2024-09-17T22:58:40.471 INFO:teuthology.orchestra.run.smithi159.stdout:5995 trunc from 0xac92baf to 0x702475e 2024-09-17T22:58:40.486 INFO:teuthology.orchestra.run.smithi159.stdout:5996 writesame 0xb5f6d2d thru 0xb5f9a3d data_size 0x1(0x2d11 bytes) 2024-09-17T22:58:40.487 INFO:teuthology.orchestra.run.smithi159.stdout:5997 trunc from 0xb5f9a3e to 0xad39fa1 2024-09-17T22:58:40.528 INFO:teuthology.orchestra.run.smithi159.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-09-17T22:58:40.532 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:40.529+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:40.604 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.604 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:194192 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 1695744~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.602084+0000", 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00032608099999999998, 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00034984399999999998, 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 194192 2024-09-17T22:58:40.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602084+0000", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602084+0000", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602085+0000", 2024-09-17T22:58:40.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6199999999999997e-07 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602089+0000", 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9060000000000004e-06 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602097+0000", 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9880000000000004e-06 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602102+0000", 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4340000000000002e-06 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602240+0000", 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000138534 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602328+0000", 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8393999999999994e-05 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602434+0000", 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000105826 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:194198 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.602244+0000", 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.00016668600000000001, 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000284257, 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.609 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 194198 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602244+0000", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602244+0000", 2024-09-17T22:58:40.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602245+0000", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4300000000000001e-07 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602249+0000", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5349999999999998e-06 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602261+0000", 2024-09-17T22:58:40.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2204999999999999e-05 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602269+0000", 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2080000000000003e-06 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602473+0000", 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020442 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602526+0000", 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3248000000000002e-05 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:194200 3.16 3.beda3896 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.602368+0000", 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.2573e-05, 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022872300000000001, 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "queued for pg", 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.613 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 194200 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602368+0000", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602368+0000", 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.614 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602369+0000", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9700000000000005e-07 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602377+0000", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7400000000000004e-06 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602387+0000", 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0001e-05 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.615 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.602395+0000", 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8759999999999994e-06 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 3 2024-09-17T22:58:40.616 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.617 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:40.661 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:40.658+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:40.730 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:40.730 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.733 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:40.731+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:40.738 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:40.834 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:40.832+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:40.888 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:40.888 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:40.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 97.024497897000003, 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:40.889 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:40.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:40.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.797658769999998, 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:40.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:40.900 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.797640243999993, 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:40.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:40.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.797622071999996, 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:40.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:40.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.797604243999999, 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.914 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 69.797289110999998, 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:40.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3154647769999999, 2024-09-17T22:58:40.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:58:40.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3154551489999999, 2024-09-17T22:58:40.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.925 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:58:40.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.929 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3154458010000001, 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:58:40.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315435879, 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:40.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:58:40.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315422423, 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:58:40.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315412611, 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.944 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:40.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3153822420000001, 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:40.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:58:40.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:58:40.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3153699190000001, 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:58:40.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:40.957 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315360412, 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:40.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:58:40.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:40.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3153503710000001, 2024-09-17T22:58:40.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:40.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315341087, 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:58:40.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.972 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.315331593, 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:58:40.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3153203250000001, 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.977 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:58:40.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3153109199999999, 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:40.982 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:40.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:40.986 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:40.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:40.988 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:40.985+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:41.089 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:41.086+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:41.139 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:41.140 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:41.140 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:41.140 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:41.149 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:41.190 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:41.188+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:41.291 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:41.289+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:41.294 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:41.294 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:41.294 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:41.294 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:41.294 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:41.302 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:41.392 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:41.389+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:41.405 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:41.405 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:41.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:41.405 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:58:41.405 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:41.414 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:41.493 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:41.490+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:41.523 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:41.523 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:41.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:41.524 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:41.532 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:41.563 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:41.616 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:41.614+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:41.678 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:41.679 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:41.679 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:41.679 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:41.679 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:41.687 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:41.694 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:41.695 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:41.692+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:41.711 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:41.712 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:41.712 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:41.730 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:41.730 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:41.747 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:41.747 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:41.763 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:41.764 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:41.780 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:41.780 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:41.796 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:41.793+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:41.797 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:41.800 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:41.801 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:41.801 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:41.801 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:41.810 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:41.897 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:41.894+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:41.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:41.997 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:41.995+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:42.066 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.066 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:42.098 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:42.095+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:42.107 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:58:42.190 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.191 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.199 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:42.196+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:42.200 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:42.300 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:42.297+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:42.318 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.318 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.327 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:42.411 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:42.408+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:42.471 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.471 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.479 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:42.502 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:42.499+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:42.602 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:42.600+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:42.623 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.623 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.632 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:42.704 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:42.702+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:42.737 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.737 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.745 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:42.805 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:42.803+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:42.849 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:42.849 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:42.857 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:42.906 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:42.904+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:43.002 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.003 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.007 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:43.004+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:43.011 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:43.108 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:43.105+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:43.119 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:58:43.160 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.160 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:43.210 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:43.207+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:43.309 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:43.306+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:43.313 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.313 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.322 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:43.410 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:43.407+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:43.466 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.467 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.475 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:43.516 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:43.514+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:43.612 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:43.610+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:43.619 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.619 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:43.713 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:43.710+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:43.733 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.733 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.742 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:43.801 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:43.826 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:43.823+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:43.886 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:43.886 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:43.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:43.915 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:43.912+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:43.933 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:43.934 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:43.950 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:43.950 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:43.950 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:43.967 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:43.967 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:43.983 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:43.984 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:44.000 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:44.000 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:44.016 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:44.013+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:44.017 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:44.033 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:44.033 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:44.036 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:44.037 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:44.040 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:44.040 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:44.040 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:44.048 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:44.118 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:58:44.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:44.119 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T22:58:44.119 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.160358 2024-09-17T22:58:44.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.16035773448469226 2024-09-17T22:58:44.132 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:44.129+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:44.200 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:44.200 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:44.200 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:44.200 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:44.200 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:44.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:44.218 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:44.215+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:44.319 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:44.316+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:44.355 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:44.355 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:44.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:44.355 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [] 2024-09-17T22:58:44.355 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:44.363 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:44.420 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:44.418+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:44.468 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:44.468 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:44.468 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:44.477 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:44.522 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:44.519+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:44.620 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:44.620 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:44.620 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:44.620 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:44.620 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:44.623 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:44.621+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:44.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:44.724 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:44.721+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:44.733 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:44.733 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:44.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:44.733 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 68.464890991000004, 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:44.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:44.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 68.464551976999999, 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:44.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:44.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.286912981999997, 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:44.742 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:44.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642722735, 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.749 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:58:44.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642700105999999, 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:58:44.753 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:58:44.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.64250002, 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:58:44.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:58:44.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:58:44.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642474448000002, 2024-09-17T22:58:44.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:44.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:58:44.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642293397, 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.767 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:44.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:58:44.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.771 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642257967999999, 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.772 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:44.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.776 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.642159067000001, 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:58:44.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.641982906999999, 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.781 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:58:44.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:58:44.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:58:44.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:58:44.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.641923411, 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.787 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:44.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:58:44.791 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.631650241999999, 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.792 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:58:44.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:58:44.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.631562066000001, 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:58:44.796 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:44.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:58:44.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.631509087000001, 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:58:44.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:58:44.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:58:44.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.631451123000002, 2024-09-17T22:58:44.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:44.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:58:44.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.631166836999999, 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:58:44.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.817 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.630990654000001, 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:58:44.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:58:44.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.630311516999999, 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.822 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:44.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.630288043, 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:58:44.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:44.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:44.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:44.831 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:44.832 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:44.833 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:44.830+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:44.933 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:44.931+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:44.979 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:44.979 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:44.979 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:44.987 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:45.034 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:45.032+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:45.091 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:45.091 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:45.091 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:45.091 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:45.091 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:45.100 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:45.131 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.7 primary-affinity to 0.160358 (8105092) 2024-09-17T22:58:45.135 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:45.132+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:45.139 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:58:45.236 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:45.233+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:45.239 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198653661000002, 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.240 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.242 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.244 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198633246, 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:45.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:45.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.248 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198612186999998, 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.249 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:45.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.253 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198589082000002, 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:45.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.255 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.256 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:45.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198566026000002, 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.258 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:45.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:45.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198535393, 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.263 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198511416999999, 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.268 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:45.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.272 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198481238999999, 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:45.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:45.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198462262, 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:45.277 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:45.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.281 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.198023351, 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:45.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:45.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.197734745999998, 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:45.286 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:45.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:45.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.197107842000001, 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.291 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.197073403000001, 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.295 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:45.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:45.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.196685243000001, 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:45.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:45.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.194905111000001, 2024-09-17T22:58:45.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:45.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.194890843, 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.309 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:45.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.313 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.194880387000001, 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:45.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.194862105999999, 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.318 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.322 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.194442638000002, 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:45.326 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 29.193816392999999, 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.327 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:45.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:45.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.331 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:45.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:45.337 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:45.335+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:45.438 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:45.436+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:45.478 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:45.478 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:45.478 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:45.487 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:45.570 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:45.568+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:45.631 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:45.631 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:45.632 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:45.632 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:45.632 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:45.640 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:45.641 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:45.638+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:45.741 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:45.738+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:45.785 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 46.628894901000002, 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:45.786 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:45.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:45.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277106502000002, 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:45.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:45.797 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277085792000001, 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.798 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:45.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:45.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277065997000001, 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.803 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:45.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.807 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277046753999997, 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:45.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277025614000003, 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:45.812 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.817 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.277000524999998, 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:45.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276981182, 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.822 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:45.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276959159, 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:45.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:45.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276865018000002, 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:45.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:45.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276838245999997, 2024-09-17T22:58:45.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:45.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:45.838 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276817719999997, 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:45.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.842 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276796427000001, 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:45.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276767552000003, 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.847 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:45.850 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276726472999997, 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.851 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:45.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:45.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.276702450000002, 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.855 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:45.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.391959823000001, 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.859 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.863 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.391938195000002, 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.867 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.391917011, 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.868 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 28.391895931000001, 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:45.872 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:45.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.876 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:45.877 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:45.877 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:45.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:45.878 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:45.876+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:45.979 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:45.977+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:45.994 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:45.994 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3.aebca74f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.85735032899999997, 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.85737189999999996, 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for peered", 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:45.995 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:45.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T22:58:45.999 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-17T22:58:45.999 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:46.004 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:46.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:46.088 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:46.086+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:46.127 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:58:46.154 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:46.154 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:46.154 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:46.154 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:46.154 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:46.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:46.176 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:46.177 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:46.181 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:46.178+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:46.196 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:46.196 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:46.196 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:46.214 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:46.215 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:46.233 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:46.233 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:46.250 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:46.250 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:46.267 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:46.267 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:46.282 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:46.280+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:46.283 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:46.283 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:46.288 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:46.288 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:46.311 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:46.311 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:46.320 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:46.383 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:46.380+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:46.464 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:46.464 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:46.472 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:46.516 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:46.514+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:46.575 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:46.575 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:46.584 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:46.625 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:46.623+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:46.686 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:46.684+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:46.727 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:46.727 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:46.736 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:46.820 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:46.818+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:46.880 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:46.880 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:46.889 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:46.890 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:46.887+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:46.990 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:46.987+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:47.034 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.034 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:47.091 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:47.089+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:47.152 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.153 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.164 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:47.205 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:47.203+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:47.293 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:47.291+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:47.314 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.315 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:47.407 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:47.405+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:47.466 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.466 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:47.515 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:47.513+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:47.595 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:47.593+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:47.618 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.618 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:47.696 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:47.694+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:47.743 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.743 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.752 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:47.797 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:47.795+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:47.897 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:47.898 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:47.898 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:47.896+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:47.907 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:47.999 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:47.996+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:48.010 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:48.010 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:48.019 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:48.100 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:48.098+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:48.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:48.170 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:48.171 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:48.181 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:48.224 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:48.221+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:48.289 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:48.303 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:48.300+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:48.332 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:48.332 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:48.342 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:48.403 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:48.401+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:48.413 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:48.414 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:48.430 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:48.431 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:48.431 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:48.446 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:48.447 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.409035496999998, 2024-09-17T22:58:48.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:48.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:48.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:48.455 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.409015082000003, 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:48.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408994022999998, 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408970918000001, 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408947861999998, 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408917228999996, 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408893253000002, 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408863074999999, 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408844098000003, 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408405187, 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.408116581999998, 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.407489677999997, 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:48.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.510 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.407455239000001, 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.511 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.512 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:48.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.407067079000001, 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.516 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:48.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:48.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:48.520 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.405286947, 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:48.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:48.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.405272678999999, 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.525 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.405262223000001, 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.530 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:48.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.534 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.405243941999998, 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:48.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:48.539 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.404824474000002, 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:48.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.544 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 32.404198229000002, 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:48.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:48.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:48.549 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:48.551 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:48.551 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:48.552 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:48.552 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:48.553 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:48.553 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:48.554 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:48.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:48.554 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:48.548+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:48.651 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:48.648+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:48.697 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:48.697 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:48.697 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:48.706 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:48.752 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:48.750+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:48.847 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:48.848 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:48.848 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:48.848 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:48.848 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:48.853 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:48.851+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:48.856 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:48.954 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:48.951+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:49.001 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 49.844988209999997, 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.002 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:49.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.003 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:49.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:49.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:49.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:49.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:49.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493199810999997, 2024-09-17T22:58:49.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:49.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:49.017 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493179101000003, 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:49.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493159306000003, 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.024 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:49.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493140062999998, 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.030 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:49.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.034 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493118922999997, 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:49.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.039 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493093834, 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:49.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:49.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493074491000002, 2024-09-17T22:58:49.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:49.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.493052468000002, 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.049 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492958326999997, 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:49.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:49.057 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492931554999998, 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:49.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:49.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.063 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492911028999998, 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:49.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:49.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492889736000002, 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.068 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.071 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492860860999997, 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:49.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:49.076 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492819781999998, 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 35.492795759000003, 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:49.081 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:49.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.608053131999998, 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:49.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:49.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:49.090 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.608031504, 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:49.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:49.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.608010319999998, 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 31.607989239999998, 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:49.100 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:49.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:49.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.104 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.105 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:49.105 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:49.105 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:49.106 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:49.107 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:49.103+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:49.156 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:58:49.207 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:49.205+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:49.219 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:49.219 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:49.219 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:49.229 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:49.309 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:49.306+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:49.335 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:49.335 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:49.335 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:49.343 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:49.410 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:49.408+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:49.448 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.059536661999999, 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:49.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:49.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.453 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.020563709999998, 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.020542933999998, 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.459 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:49.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.463 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.020524143999999, 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:49.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:49.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.020478933, 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:49.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:49.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:49.473 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.020445105, 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3155278379999995, 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.478 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:58:49.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3155073700000006, 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:58:49.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3154858090000001, 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:58:49.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:58:49.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.315465992, 2024-09-17T22:58:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:58:49.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:58:49.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.315446004, 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:49.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189941 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3148841460000007, 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060996437000000001, 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189941 2024-09-17T22:58:49.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129927+0000", 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129931+0000", 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1409999999999998e-06 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129947+0000", 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6359999999999999e-05 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129955+0000", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1790000000000002e-06 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190765+0000", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060810757999999999 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190831+0000", 2024-09-17T22:58:49.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5188e-05 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190923+0000", 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2139999999999995e-05 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3147805909999999, 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:58:49.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3147578459999991, 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:58:49.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:58:49.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3147367130000003, 2024-09-17T22:58:49.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:58:49.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:58:49.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3147165960000002, 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:58:49.521 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:58:49.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:58:49.525 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3146893859999995, 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:58:49.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3145185250000004, 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:58:49.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.3144942069999992, 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:58:49.535 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:58:49.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:58:49.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.3092202049999999, 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:58:49.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:49.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.546 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:49.548 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:49.548 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:49.546+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:49.649 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:49.647+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:49.693 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:49.693 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:49.693 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:49.702 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:49.750 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:49.747+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:49.805 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:49.805 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:49.805 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:49.814 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:49.855 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:49.853+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:49.952 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:49.950+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:49.958 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:49.958 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702878705000003, 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:49.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.963 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702862699999997, 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:49.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702829930000007, 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:49.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:49.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702802192000007, 2024-09-17T22:58:49.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:49.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702779773000003, 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:49.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702757493000007, 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:49.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:49.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702550087999995, 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:49.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:49.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.702003473000005, 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.992 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:49.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.700932277000007, 2024-09-17T22:58:49.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:49.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:49.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.000 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.700920822, 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.004 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:50.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.700909844999998, 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:50.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.698835712999994, 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:50.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.698823399000005, 2024-09-17T22:58:50.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:50.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.698810162000001, 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:50.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.698799332999997, 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.028 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:50.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:50.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.696840457999997, 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:50.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.696816100000007, 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:50.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.696786881999998, 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.042 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.696020313999995, 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.047 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:50.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.051 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:50.052 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:50.052 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 73.696007211999998, 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:50.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.057 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:50.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:50.059 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:50.056+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:50.147 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:58:50.147 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:50.148 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-17T22:58:50.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-17T22:58:50.159 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:50.157+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:50.209 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:50.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:50.210 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:50.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:50.300 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:50.298+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:50.359 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:50.360 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:50.360 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:50.360 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:50.360 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:50.360 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:50.358+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:50.368 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:50.460 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-09-17T22:58:50.457+0000 7fe9a1abd640 -1 mon.a@0(leader).osd e103 definitely_dead 0 2024-09-17T22:58:50.461 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:50.458+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:50.513 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:50.513 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:50.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:50.513 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.649725135, 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:50.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:50.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:50.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.422886008000006, 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.521 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:50.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:50.525 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.422867482000001, 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:50.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:50.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.422849310000004, 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:50.535 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.422831482000007, 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:50.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 79.422516349000006, 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.540 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:50.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:50.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940692015, 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:58:50.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:58:50.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:50.549 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940682387000001, 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:50.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:58:50.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940673039, 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:58:50.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940663117, 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:58:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940649661, 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:58:50.563 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:50.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:58:50.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.568 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940639849, 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:58:50.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:58:50.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.573 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940609479999999, 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:50.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:58:50.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940597156999999, 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.578 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:58:50.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940587649999999, 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.583 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:58:50.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:50.587 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940577609, 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:58:50.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:58:50.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940568324999999, 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.592 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:58:50.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:58:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940558831000001, 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:58:50.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:58:50.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:50.601 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940547562999999, 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:58:50.604 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.940538158000001, 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:50.606 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:58:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:58:50.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:50.611 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:50.611 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:50.613 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:50.613 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:50.610+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:50.714 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:50.711+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:50.732 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:50.732 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:50.749 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:50.749 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:50.749 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:50.757 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:50.757 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:50.757 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:50.757 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:50.765 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:50.765 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:50.766 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:50.781 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:50.781 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:50.798 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:50.798 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:50.814 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:50.814 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:50.815 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:50.812+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:50.832 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:50.834 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:50.835 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:50.872 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:50.872 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:50.881 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:50.925 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:50.923+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:50.984 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:50.985 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:50.993 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:51.037 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:51.035+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:51.097 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.097 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.105 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:51.117 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:51.115+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:51.173 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:58:51.178 INFO:teuthology.orchestra.run.smithi159.stderr:marked in osd.5. 2024-09-17T22:58:51.196 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-17T22:58:51.218 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:51.215+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:51.258 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.258 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.267 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:51.318 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:51.316+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:51.373 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.373 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.382 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:51.423 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:51.421+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:51.519 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:51.517+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:51.525 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.526 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.534 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:51.621 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:51.618+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:51.677 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.677 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.686 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:51.721 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:51.719+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:51.823 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:51.820+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:51.829 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.829 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.838 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:51.924 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:51.922+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:51.982 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:51.982 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:51.990 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:52.025 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:52.023+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:52.126 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:52.124+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:52.133 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.134 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.142 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:52.180 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:58:52.227 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:52.225+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:52.252 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.252 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.261 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:52.328 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:52.325+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:52.374 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.375 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.383 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:52.429 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:52.427+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:52.486 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.487 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.495 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:52.536 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:52.534+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:52.631 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:52.629+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:52.638 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.638 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.647 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:52.733 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:52.731+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:52.750 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.750 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:52.834 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:52.832+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:52.835 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:52.903 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:52.903 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:52.911 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:52.935 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:52.933+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:52.958 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:52.959 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:52.975 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:52.976 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:52.976 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:52.991 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:52.992 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:53.008 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:53.008 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:53.015 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:53.025 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:53.025 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:53.026 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:53.041 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:53.042 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:53.059 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:53.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:53.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:53.070 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:53.068+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:53.130 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.130 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:53.130 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:53.130 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:53.136 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:53.134+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:53.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:53.237 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:53.235+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:53.318 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.318 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:53.319 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:53.319 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:53.319 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:53.328 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:53.340 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:53.338+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:53.440 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:53.438+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.448 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.58446139199999, 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:58:53.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:58:53.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:53.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.357622265000003, 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:58:53.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:58:53.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:58:53.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.357603738999998, 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:53.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:58:53.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.357585567000001, 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.465 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:58:53.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:58:53.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.357567739000004, 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:58:53.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:58:53.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.357252606000003, 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:58:53.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.480 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.480 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875428272000001, 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:58:53.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875418644, 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:58:53.485 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:58:53.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:58:53.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875409296000001, 2024-09-17T22:58:53.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875399374000001, 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:58:53.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875385917999999, 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:53.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:58:53.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875376106000001, 2024-09-17T22:58:53.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:58:53.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.510 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875345737, 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.511 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:58:53.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:53.515 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875333414, 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:53.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:58:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875323907, 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.521 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:58:53.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:58:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875313866000001, 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.526 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:58:53.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.530 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875304582, 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:58:53.535 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875295088, 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.536 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:58:53.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.87528382, 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.541 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:58:53.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:58:53.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:53.545 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.875274415, 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:58:53.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.549 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:53.550 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:53.550 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:53.551 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:53.552 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:53.549+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:53.652 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:53.649+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:53.702 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:53.703 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:53.703 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:53.712 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:53.755 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:53.753+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:53.824 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.824 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:53.824 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:53.824 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:53.824 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:53.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:53.874 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:53.872+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:53.955 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:53.953+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:53.998 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:53.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197496 3.17 e106/105 3:ec809d31:::rbd_data.1188eaa36ced.0000000000000003:head v 106'608, mlcod=0'0)", 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.256793+0000", 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73780397399999997, 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001538595, 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256793+0000", 2024-09-17T22:58:53.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256793+0000", 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256794+0000", 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7000000000000001e-07 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256799+0000", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3890000000000004e-06 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256812+0000", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3410999999999999e-05 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256821+0000", 2024-09-17T22:58:54.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.242e-06 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256899+0000", 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8263000000000006e-05 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.256923+0000", 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3640000000000001e-05 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258261+0000", 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013380309999999999 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258332+0000", 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1348999999999997e-05 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197505 3.d e106/105 3:b1c60735:::rbd_data.1188eaa36ced.0000000000000024:head v 106'357, mlcod=0'0)", 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.258075+0000", 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73652247500000001, 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016153910000000001, 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258075+0000", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258075+0000", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258075+0000", 2024-09-17T22:58:54.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9999999999999999e-07 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258079+0000", 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9740000000000004e-06 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258090+0000", 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0752e-05 2024-09-17T22:58:54.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258097+0000", 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0380000000000002e-06 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258323+0000", 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022554699999999999 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.258361+0000", 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8649999999999998e-05 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259608+0000", 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012469250000000001 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259690+0000", 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2205000000000003e-05 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197510 3.c e106/105 3:30ba1bc1:::rbd_data.1188eaa36ced.0000000000000042:head v 106'362, mlcod=0'0)", 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.259528+0000", 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73506905700000003, 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015595019999999999, 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259528+0000", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259528+0000", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259529+0000", 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259534+0000", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1959999999999999e-06 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259560+0000", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6383000000000001e-05 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259567+0000", 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8990000000000004e-06 2024-09-17T22:58:54.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260132+0000", 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00056475899999999996 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260152+0000", 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9715999999999999e-05 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261051+0000", 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000899719 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261088+0000", 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6424000000000002e-05 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197512 3.2 e106/105 3:43555f33:::rbd_data.1188eaa36ced.000000000000004e:head v 106'800, mlcod=0'0)", 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.259584+0000", 2024-09-17T22:58:54.013 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73501357499999997, 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017835279999999999, 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259584+0000", 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259584+0000", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259584+0000", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9699999999999996e-07 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259589+0000", 2024-09-17T22:58:54.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5249999999999999e-06 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259595+0000", 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9580000000000004e-06 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259599+0000", 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.459e-06 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259660+0000", 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1017000000000002e-05 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259681+0000", 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0661000000000001e-05 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261326+0000", 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0016453629999999999 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261367+0000", 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1047999999999997e-05 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.018 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197514 3.5 e106/105 3:a1d9fe15:::rbd_data.1188eaa36ced.00000000000000ba:head v 106'664, mlcod=0'0)", 2024-09-17T22:58:54.020 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.259956+0000", 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73464145599999997, 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001648381, 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259956+0000", 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259956+0000", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259980+0000", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4184000000000001e-05 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.259996+0000", 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6147e-05 2024-09-17T22:58:54.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260006+0000", 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3309999999999993e-06 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260011+0000", 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8320000000000002e-06 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260217+0000", 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020516299999999999 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.260238+0000", 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0982e-05 2024-09-17T22:58:54.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261569+0000", 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013312949999999999 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261604+0000", 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5447000000000001e-05 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:197518 3.16 e106/105 3:6c767cef:::rbd_data.1188eaa36ced.000000000000014a:head v 106'1208, mlcod=0'0)", 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.261087+0000", 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.73351057399999997, 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001243039, 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261087+0000", 2024-09-17T22:58:54.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261087+0000", 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261088+0000", 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0100000000000004e-07 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261093+0000", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6180000000000003e-06 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261104+0000", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0879999999999999e-05 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261111+0000", 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5509999999999996e-06 2024-09-17T22:58:54.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261173+0000", 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2371000000000003e-05 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.261188+0000", 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5102e-05 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:54.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.262291+0000", 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011025690000000001 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.262330+0000", 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9246999999999999e-05 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.11 106/105 [PushOp(3:8e52d0d6:::rbd_data.11883e449d95.0000000000000004:head, version: 70'313, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8e52d0d6:::rbd_data.11883e449d95.0000000000000004:head@70'313, size: 5505024, copy_subset: [(0, 5505024)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.327424+0000", 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.66717320999999996, 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011797019999999999, 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.031 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327424+0000", 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327424+0000", 2024-09-17T22:58:54.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327425+0000", 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4700000000000001e-07 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327823+0000", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00039820699999999998 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327840+0000", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.717e-05 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327853+0000", 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2984e-05 2024-09-17T22:58:54.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327929+0000", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5245000000000002e-05 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.327938+0000", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.328604+0000", 2024-09-17T22:58:54.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066610900000000002 2024-09-17T22:58:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1c 106/105 [PushOp(3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:head, version: 42'87, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:head@42'87, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.56022973499999995, 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020122199999999999, 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3e-07 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435461+0000", 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010930320000000001 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435479+0000", 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8026999999999999e-05 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435490+0000", 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0681e-05 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435569+0000", 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8534000000000004e-05 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435582+0000", 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3020000000000001e-05 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.436380+0000", 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079829599999999999 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1c 106/105 [PushOp(3:3ce6a303:::rbd_data.118827e1bedb.0000000000000000:26, version: 76'371, data_included: [0~655360,1245184~65536,4194304~196608], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:3ce6a303:::rbd_data.118827e1bedb.0000000000000000:26@76'371, size: 7340032, copy_subset: [(0, 7340032)], clone_subset: {}, snapset: 26=[]:{26: [26]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7340032, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.524955+0000", 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.46964218400000002, 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017962900000000001, 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.524955+0000", 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.524955+0000", 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.524956+0000", 2024-09-17T22:58:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:58:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.525901+0000", 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094516999999999995 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.525916+0000", 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5519000000000002e-05 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.525928+0000", 2024-09-17T22:58:54.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1484000000000001e-05 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.525986+0000", 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8391999999999999e-05 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.525994+0000", 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0700000000000007e-06 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.526751+0000", 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075716900000000003 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.049 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.41581918899999998, 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019877050000000002, 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.050 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578779+0000", 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0200000000000002e-07 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579745+0000", 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096608000000000002 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579762+0000", 2024-09-17T22:58:54.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7058000000000002e-05 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579772+0000", 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0505e-05 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579840+0000", 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7631000000000004e-05 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579848+0000", 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3650000000000004e-06 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.580766+0000", 2024-09-17T22:58:54.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000917464 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:58:54.058 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.41210699299999998, 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00207756, 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582491+0000", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583432+0000", 2024-09-17T22:58:54.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094108400000000002 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583449+0000", 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6932999999999999e-05 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583460+0000", 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583518+0000", 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7924000000000003e-05 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583526+0000", 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.584568+0000", 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010420010000000001 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.063 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:33b5f7db:::rbd_data.1188aa1637fb.0000000000000000:head, version: 47'64, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:33b5f7db:::rbd_data.1188aa1637fb.0000000000000000:head@47'64, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.378827362, 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018244229999999999, 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.064 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615771+0000", 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616737+0000", 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096671699999999997 2024-09-17T22:58:54.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616755+0000", 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7660999999999999e-05 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616768+0000", 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2843000000000001e-05 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616834+0000", 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6478999999999998e-05 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616843+0000", 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3780000000000001e-06 2024-09-17T22:58:54.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.617594+0000", 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075164899999999996 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.069 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:308dd7b1:::rbd_data.118894cf7479.0000000000000003:18, version: 55'96, data_included: [0~524288,536576~4096,544768~262144], data_size: 790528, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:308dd7b1:::rbd_data.118894cf7479.0000000000000003:18@55'96, size: 811008, copy_subset: [(0, 811008)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 811008, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.629150+0000", 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.36544691200000001, 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001571074, 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629150+0000", 2024-09-17T22:58:54.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629150+0000", 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629151+0000", 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.13e-07 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629853+0000", 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070170599999999997 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629871+0000", 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7745e-05 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629881+0000", 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0767e-05 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629943+0000", 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1383000000000005e-05 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.629951+0000", 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1699999999999997e-06 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.630722+0000", 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077068999999999998 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.24247485099999999, 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:58:54.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a, version: 34'124, data_included: [16384~163840,212992~163840,409600~163840,606208~163840,802816~163840,999424~49152,17782901~10673,19576767~5650], data_size: 884675, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a@34'124, size: 19582417, copy_subset: [(0, 19582417)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19582417, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.23779419800000001, 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021507340000000001, 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:58:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756813+0000", 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757692+0000", 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087948200000000001 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757727+0000", 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5073000000000001e-05 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757750+0000", 2024-09-17T22:58:54.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3201e-05 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757830+0000", 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9196000000000005e-05 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757838+0000", 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9409999999999996e-06 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.758954+0000", 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011154310000000001 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f, version: 42'187, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f@42'187, size: 17301504, copy_subset: [(0, 17301504)], clone_subset: {}, snapset: f=[]:{f: [f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.778197+0000", 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.21640029799999999, 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017784229999999999, 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.086 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.778197+0000", 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.778197+0000", 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.778198+0000", 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5799999999999999e-07 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779172+0000", 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097420500000000001 2024-09-17T22:58:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779194+0000", 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1967000000000002e-05 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779210+0000", 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5815e-05 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779307+0000", 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7207999999999995e-05 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779315+0000", 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4400000000000005e-06 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.779976+0000", 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00066012999999999998 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f, version: 42'187, data_included: [1048576~550078,2323151~47926,5670009~32074,12058624~28972,12091669~23433,15900451~27488], data_size: 709971, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f@42'187, size: 17301504, copy_subset: [(0, 17301504)], clone_subset: {}, snapset: f=[]:{f: [f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 17301504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:58:54.091 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.213265292, 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001839233, 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:58:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781333+0000", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782009+0000", 2024-09-17T22:58:54.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000676251 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782031+0000", 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1691e-05 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782067+0000", 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5867000000000001e-05 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782131+0000", 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.478e-05 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782140+0000", 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.783171+0000", 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001031541 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.096 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fe5a6720:::rbd_data.1188102a4abc.0000000000000001:17, version: 54'246, data_included: [156657~61956,930379~39275,4664382~9996,11713353~17630,12422072~15976], data_size: 144833, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fe5a6720:::rbd_data.1188102a4abc.0000000000000001:17@54'246, size: 12438048, copy_subset: [(0, 12438048)], clone_subset: {}, snapset: 17=[]:{17: [17]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 12438048, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.803166+0000", 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.191431042, 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012998300000000001, 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.097 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803166+0000", 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803166+0000", 2024-09-17T22:58:54.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803167+0000", 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9299999999999997e-07 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803292+0000", 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012536599999999999 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803317+0000", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4133000000000001e-05 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803338+0000", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1744000000000001e-05 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803421+0000", 2024-09-17T22:58:54.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2725000000000005e-05 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.803435+0000", 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4384e-05 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.804466+0000", 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001030785 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fa49e8c6:::rbd_data.11883e449d95.0000000000000000:head, version: 70'338, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fa49e8c6:::rbd_data.11883e449d95.0000000000000000:head@70'338, size: 16674582, copy_subset: [(0, 16674582)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.835640+0000", 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.158956982, 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001706094, 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.835640+0000", 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.835640+0000", 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.835641+0000", 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.836620+0000", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097944099999999995 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.836636+0000", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5310000000000001e-05 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.836645+0000", 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7569999999999999e-06 2024-09-17T22:58:54.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.836712+0000", 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6403999999999996e-05 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.836720+0000", 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4889999999999995e-06 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.837347+0000", 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00062624899999999995 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.d 106/105 [PushOp(3:b068c059:::rbd_data.1188aa1637fb.0000000000000001:head, version: 47'91, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b068c059:::rbd_data.1188aa1637fb.0000000000000001:head@47'91, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.922133+0000", 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.072463954999999997, 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001560748, 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:54.106 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922133+0000", 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922133+0000", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922134+0000", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4900000000000004e-07 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922836+0000", 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070116799999999995 2024-09-17T22:58:54.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922853+0000", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7448e-05 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922864+0000", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0967e-05 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922926+0000", 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2102999999999998e-05 2024-09-17T22:58:54.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.922935+0000", 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8170000000000005e-06 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.923694+0000", 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075929600000000002 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:54.112 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:54.112 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:54.113 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:54.111+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:54.113 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:54.197 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:58:54.236 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:54.236 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:54.236 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:54.237 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:54.237 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:54.234+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:54.247 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:54.317 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:54.315+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:54.408 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:54.408 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:54.408 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:54.408 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:54.408 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:54.418 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:54.418 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:54.415+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:54.521 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:54.518+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 78.256206098999996, 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:58:54.525 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:54.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:58:54.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 78.255867085000006, 2024-09-17T22:58:54.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:58:54.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:58:54.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:58:54.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 70.078228089999996, 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:58:54.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.541 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.434037842999999, 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:58:54.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:58:54.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.434015213999999, 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:58:54.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:58:54.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433815127999999, 2024-09-17T22:58:54.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:58:54.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.554 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433789556000001, 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.555 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:58:54.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:58:54.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433608505000002, 2024-09-17T22:58:54.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.563 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433573076000002, 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.564 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433474175000001, 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:58:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:58:54.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:58:54.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433298014999998, 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:58:54.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.433238519, 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.577 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:58:54.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:58:54.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.581 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422965349999998, 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:58:54.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:58:54.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422877174, 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:58:54.586 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:58:54.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.590 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422824194999997, 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:58:54.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:58:54.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422766230999997, 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422481945000001, 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.600 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:58:54.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:58:54.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.422305762000001, 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:58:54.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.421626625000002, 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.609 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.613 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.421603150999999, 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:58:54.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:58:54.617 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:54.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:54.618 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:54.619 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:54.661 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:54.658+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:54.722 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:54.720+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:54.771 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:54.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:54.771 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:54.771 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:54.780 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:54.823 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:54.820+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:54.865 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:58:54.924 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:54.922+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:54.945 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:54.945 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:54.945 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:54.945 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:54.945 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:54.954 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:55.024 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:55.022+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:55.058 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:55.058 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:55.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:55.058 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:55.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.017074477000001, 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.017054062, 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:55.063 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:55.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.017033003000002, 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.068 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:55.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:55.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:55.072 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.017009897999998, 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:55.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:55.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016986842000001, 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:55.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016956209, 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:55.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016932232999999, 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.086 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:55.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:55.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016902055000003, 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.091 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016883077999999, 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:55.095 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:55.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:55.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016444167000003, 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.016155562000002, 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.015528658000001, 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.015494218999997, 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.114 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.118 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.015106058999997, 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:55.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:55.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.013325926999997, 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.123 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:55.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:55.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.013311659000003, 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:55.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.131 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.132 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.013301202999997, 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:55.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.013282922000002, 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.137 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:55.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.012863453999998, 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.142 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:55.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:55.145 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:55.146 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 39.012237208999998, 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.147 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.148 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:55.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:55.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.151 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:55.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:55.153 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:55.154 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:55.150+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:55.254 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:55.252+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:197693 3.2 3.9644a922 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:54.925122+0000", 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.373200119, 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.37321957700000002, 2024-09-17T22:58:55.300 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "waiting for peered", 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 197693 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925122+0000", 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925122+0000", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925122+0000", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0800000000000004e-07 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925126+0000", 2024-09-17T22:58:55.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.315e-06 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925134+0000", 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4959999999999996e-06 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925139+0000", 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2120000000000002e-06 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925194+0000", 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4718e-05 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925204+0000", 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.009e-05 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:197700 3.2 3.fd13c122 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T22:58:55.304 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:54.925228+0000", 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 0.37309360200000002, 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.37315558700000001, 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "waiting for peered", 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 197700 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:55.305 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925228+0000", 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925228+0000", 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925229+0000", 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.5699999999999998e-07 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925231+0000", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4420000000000001e-06 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925238+0000", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6699999999999997e-06 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925243+0000", 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1830000000000002e-06 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925289+0000", 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6468000000000003e-05 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:54.925298+0000", 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8170000000000005e-06 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:58:55.308 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 2 2024-09-17T22:58:55.309 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:55.309 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:55.316 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:55.317 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:55.334 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:55.336 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:55.336 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:55.350 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:55.350 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:55.355 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:55.352+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:55.367 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:55.367 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:55.384 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:55.384 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:55.401 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:55.415 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:55.415 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:55.417 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:55.418 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:55.421 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:55.422 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:55.424 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:55.466 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:55.463+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:55.557 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:55.555+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:55.568 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:55.569 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:55.577 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:55.659 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:55.657+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:55.686 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:55.686 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:55.695 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:55.759 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:55.757+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:55.797 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:55.798 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:55.806 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:55.860 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:55.858+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:55.919 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:55.920 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:55.931 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:55.961 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:55.959+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:56.062 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:56.060+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:56.074 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.074 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.083 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:56.164 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:56.162+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:56.186 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.186 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.194 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:56.197 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:58:56.197 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:58:56.197 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T22:58:56.197 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-17T22:58:56.265 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:56.262+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:56.296 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.297 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:56.366 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:56.364+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:56.408 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.409 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.418 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:56.476 INFO:teuthology.orchestra.run.smithi159.stdout:moved 0 / 82 (0%) 2024-09-17T22:58:56.476 INFO:teuthology.orchestra.run.smithi159.stdout:avg 11.7143 2024-09-17T22:58:56.476 INFO:teuthology.orchestra.run.smithi159.stdout:stddev 2.37762 -> 2.37762 (expected baseline 3.16872) 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:max osd.0 with 13 -> 13 pgs (1.10976 -> 1.10976 * mean) 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:oload 1000 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:max_change 0.05 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:max_change_osds 4 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:average_utilization 0.0009 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stdout:overload_utilization 0.0089 2024-09-17T22:58:56.477 INFO:teuthology.orchestra.run.smithi159.stderr:no change 2024-09-17T22:58:56.489 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T22:58:56.489 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T22:58:56.502 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:56.500+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:56.562 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.562 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.570 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:56.573 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:56.571+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:56.669 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:56.667+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:56.713 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.713 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:58:56.770 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:56.768+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:moved 0 / 82 (0%) 2024-09-17T22:58:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:avg 11.7143 2024-09-17T22:58:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:stddev 2.37762 -> 2.37762 (expected baseline 3.16872) 2024-09-17T22:58:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:max osd.0 with 13 -> 13 pgs (1.10976 -> 1.10976 * mean) 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:oload 1000 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:max_change 3 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:max_change_osds 4 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:average_utilization 0.0009 2024-09-17T22:58:56.806 INFO:teuthology.orchestra.run.smithi159.stdout:overload_utilization 0.0089 2024-09-17T22:58:56.807 INFO:teuthology.orchestra.run.smithi159.stderr:no change 2024-09-17T22:58:56.818 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T22:58:56.818 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-17T22:58:56.871 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:56.869+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:56.875 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:56.875 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:56.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:58:56.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:58:56.972 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:56.969+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:57.033 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:57.033 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:57.042 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:58:57.083 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:57.081+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:57.174 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:57.172+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:57.187 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:57.187 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:57.195 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:58:57.275 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:57.273+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:57.299 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:57.299 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:57.307 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:58:57.376 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:57.373+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:57.410 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:57.411 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:57.419 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:58:57.422 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:58:57.505 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:57.503+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:57.567 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:58:57.567 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:58:57.578 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:57.575+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:57.579 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:58:57.588 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:57.588 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:57.605 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:57.622 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:57.622 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:57.640 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:57.656 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:57.656 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:57.673 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:57.678 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:57.676+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:57.689 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:57.689 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:57.692 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:57.692 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:57.694 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:57.694 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:57.694 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:57.694 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:57.703 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:58:57.786 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:57.784+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:57.846 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:57.846 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:57.855 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:58:57.867 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:58:57.872 INFO:teuthology.orchestra.run.smithi159.stdout:moved 9 / 82 (10.9756%) 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:avg 11.7143 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:stddev 2.37762 -> 2.93814 (expected baseline 3.16872) 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:max osd.0 with 13 -> 8 pgs (1.10976 -> 0.682927 * mean) 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:oload 110 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:max_change 3 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:max_change_osds 4 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:average_utilization 0.0009 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:overload_utilization 0.0010 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4 weight 1.0000 -> 0.7237 2024-09-17T22:58:57.873 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3 weight 1.0000 -> 0.7740 2024-09-17T22:58:57.874 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0 weight 1.0000 -> 0.7769 2024-09-17T22:58:57.874 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7 weight 1.0000 -> 0.8289 2024-09-17T22:58:57.881 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:57.879+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:57.890 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T22:58:57.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936571702000002, 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:57.978 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:57.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:58:57.982 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936551287, 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:57.983 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:58:57.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:58:57.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936530228000002, 2024-09-17T22:58:57.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:58:57.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:57.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936507122999998, 2024-09-17T22:58:57.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:58:57.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:57.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:57.997 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936484067000002, 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:57.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.000 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:58:58.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.000 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:58:58.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.004 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936453434000001, 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.008 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936429457999999, 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:58:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:58:58.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936399280000003, 2024-09-17T22:58:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:58:58.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.017 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.936380303, 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.018 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:58:58.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.935941391999997, 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.022 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:58:58.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.935652787000002, 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:58:58.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:58:58.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.935025883000002, 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:58:58.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.934991443999998, 2024-09-17T22:58:58.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:58.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.037 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:58:58.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.934603283999998, 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.040 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.044 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.932823151999997, 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:58:58.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.932808883999996, 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.049 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:58:58.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.932798427999998, 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:58:58.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.932780147000003, 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.058 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.059 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.063 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.932360678999999, 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.064 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:58:58.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:58:58.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:58:58.067 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.931734433999999, 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:58:58.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:58:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.072 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:58.074 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:58:58.074 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:58.071+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:58.174 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:58.172+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:58.179 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:58.179 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:58.179 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:58:58.180 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:58.188 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:58:58.216 INFO:teuthology.orchestra.run.smithi159.stdout:moved 0 / 82 (0%) 2024-09-17T22:58:58.216 INFO:teuthology.orchestra.run.smithi159.stdout:avg 11.7143 2024-09-17T22:58:58.216 INFO:teuthology.orchestra.run.smithi159.stdout:stddev 2.93814 -> 2.93814 (expected baseline 3.16872) 2024-09-17T22:58:58.216 INFO:teuthology.orchestra.run.smithi159.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:max osd.5 with 14 -> 14 pgs (1.19512 -> 1.19512 * mean) 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:oload 1000 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:max_change 1 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:max_change_osds 4 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:average_utilization 0.0009 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stdout:overload_utilization 0.0093 2024-09-17T22:58:58.217 INFO:teuthology.orchestra.run.smithi159.stderr:no change 2024-09-17T22:58:58.229 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T22:58:58.229 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T22:58:58.275 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:58.273+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:58.322 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:58.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:58:58.323 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:58:58.323 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:58:58.323 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:58.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:58:58.376 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:58.374+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:58.439 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 59.283173671999997, 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:58:58.440 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:58.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:58:58.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931385272999997, 2024-09-17T22:58:58.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:58.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:58:58.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:58:58.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931364563000002, 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.452 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:58:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.456 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931344768000002, 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.457 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:58.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931325524999998, 2024-09-17T22:58:58.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931304384999997, 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:58:58.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931279296, 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.474 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:58:58.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:58:58.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931259953000001, 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.479 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:58:58.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.483 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931237930000002, 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:58:58.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:58.488 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931143789000004, 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:58:58.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:58:58.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931117016999998, 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:58:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:58:58.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.498 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931096490999998, 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.499 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:58.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931075198000002, 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:58:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.511 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931046322999997, 2024-09-17T22:58:58.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.513 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:58.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:58:58.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:58:58.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.931005243999998, 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.519 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.521 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 44.930981221000003, 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.524 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:58:58.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:58:58.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.529 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.046238594000002, 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.530 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:58:58.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:58:58.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:58.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.046216966000003, 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:58:58.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:58:58.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.540 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.046195781999998, 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:58.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:58:58.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 41.046174702000002, 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:58:58.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:58:58.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:58:58.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.549 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.550 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:58:58.550 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:58:58.550 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:58:58.551 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:58:58.551 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:58.548+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout:moved 0 / 82 (0%) 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout:avg 11.7143 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout:stddev 2.93814 -> 2.93814 (expected baseline 3.16872) 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout:max osd.5 with 14 -> 14 pgs (1.19512 -> 1.19512 * mean) 2024-09-17T22:58:58.553 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stdout:oload 1000 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stdout:max_change 3 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stdout:max_change_osds 4 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stdout:average_utilization 0.0009 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stdout:overload_utilization 0.0093 2024-09-17T22:58:58.554 INFO:teuthology.orchestra.run.smithi159.stderr:no change 2024-09-17T22:58:58.652 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:58.650+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:58.668 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:58.668 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:58.668 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:58.668 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:58.677 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:58:58.753 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:58.750+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:58.781 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:58.781 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:58.781 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:58.781 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:58.781 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:58.790 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:58:58.874 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:58.872+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.549635487000003, 2024-09-17T22:58:58.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:58.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:58:58.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.510662535000002, 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:58.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:58:58.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.510641759000002, 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:58:58.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:58:58.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:58.952 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.510622969000003, 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:58.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.956 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.510577757999997, 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:58:58.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.961 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.510543929999997, 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.962 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:58:58.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:58:58.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.805626662999998, 2024-09-17T22:58:58.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:58.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:58:58.970 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.805606194999999, 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.971 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:58.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:58:58.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.805584633999999, 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.976 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:58:58.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:58.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.805564817, 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:58.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:58:58.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.805544828999999, 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:58:58.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.989 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189941 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804982971000001, 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060996437000000001, 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189941 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129927+0000", 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-17T22:58:58.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129931+0000", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1409999999999998e-06 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129947+0000", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6359999999999999e-05 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129955+0000", 2024-09-17T22:58:58.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1790000000000002e-06 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190765+0000", 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060810757999999999 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190831+0000", 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5188e-05 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190923+0000", 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2139999999999995e-05 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804879415999999, 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:58.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:58:58.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804856671, 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:58:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804835537999999, 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:58:59.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804815421000001, 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.008 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:58:59.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804788211000002, 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.012 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:58:59.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:58:59.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804617350000001, 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.021 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:58:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.804593032, 2024-09-17T22:58:59.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:58:59.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:59.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.799319029999999, 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:59.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:58:59.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:58:59.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.035 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:58:59.037 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:59.034+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:59.137 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:58:59.135+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:58:59.182 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:59.182 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:59.182 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.182 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3.52c09d30 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 1.3021037120000001, 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3021209490000001, 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "waiting for peered", 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.183 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:58:59.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.185 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.186 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: ], 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 1 2024-09-17T22:58:59.187 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.191 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:58:59.276 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:59.274+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:58:59.334 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:59.335 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:59.335 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.338 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:59.336+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:59.343 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:58:59.439 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:58:59.437+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233195533, 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:58:59.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.494 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233179527999994, 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:58:59.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.499 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233146758000004, 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:58:59.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233119020000004, 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:58:59.504 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:58:59.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:58:59.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233096601, 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.509 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.513 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.233074321000004, 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:58:59.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:58:59.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.232866916000006, 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.519 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:58:59.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.232320301000001, 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:58:59.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:59.528 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.231249105000003, 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:58:59.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:58:59.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:58:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.533 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.231237649999997, 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.231226672999995, 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:58:59.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:58:59.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.542 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.229152541000005, 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.543 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:58:59.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:58:59.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:59.547 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.229140227000002, 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:58:59.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.229126989999997, 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.552 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:58:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:58:59.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.229116160999993, 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.557 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:58:59.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.227157285999994, 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.562 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.564 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:58:59.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:59.566 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.227132928000003, 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.569 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.227103709999994, 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:58:59.571 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:58:59.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16143 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.226337142000006, 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050458559, 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16143 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:59.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259648+0000", 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7099999999999998e-07 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259650+0000", 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.201e-06 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259654+0000", 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9859999999999996e-06 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259658+0000", 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309946+0000", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050288658999999999 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310021+0000", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4648999999999996e-05 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310107+0000", 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5733000000000001e-05 2024-09-17T22:58:59.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 83.226324039999994, 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:58:59.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:58:59.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:58:59.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:58:59.585 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.587 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:58:59.587 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:58:59.585+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:58:59.688 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:58:59.686+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:58:59.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:58:59.731 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:59.732 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:59.732 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:58:59.732 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.740 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:58:59.788 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:58:59.786+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:58:59.826 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:58:59.827 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:58:59.843 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:58:59.844 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:58:59.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:58:59.860 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:58:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:58:59.879 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:58:59.879 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:58:59.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:58:59.890 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:58:59.888+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:58:59.910 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:58:59.910 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:58:59.932 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:58:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:58:59.965 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:58:59.965 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:58:59.970 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:58:59.970 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:58:59.990 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:58:59.988+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:00.031 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.031 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.043 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:00.092 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:00.090+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:00.192 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:00.191+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:00.216 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.216 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.228 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:00.293 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:00.292+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:00.349 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.350 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:00.395 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:00.393+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:00.472 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.472 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.482 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:00.525 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:00.524+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:00.585 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.585 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.593 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:00.596 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:00.594+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:00.697 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:00.694+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:00.736 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.737 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.745 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:00.797 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:00.795+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:00.888 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:00.889 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:00.889 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:00.898 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:00.896+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:00.905 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:01.000 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:00.997+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:01.008 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.009 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.017 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:01.100 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:01.098+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:01.120 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.120 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.129 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:01.201 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:01.199+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:01.232 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.232 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.241 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:01.302 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:01.300+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:01.385 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.385 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.394 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:01.404 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:01.402+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:01.497 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.498 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.505 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:01.503+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:01.507 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:01.606 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:01.604+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:01.610 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.610 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:01.707 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:01.705+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:01.733 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.733 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.743 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:01.808 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:01.806+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:01.846 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:01.847 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:01.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:01.939 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:01.937+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:01.971 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:02.011 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:02.011 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:02.015 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:02.013+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:02.020 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:02.096 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:02.097 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:02.110 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:02.107+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:02.112 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:02.113 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:02.113 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:59:02.129 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:02.129 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:02.145 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:02.146 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:02.162 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:02.162 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:02.165 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:02.165 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:02.175 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:02.177 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:02.178 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:02.194 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:02.194 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:02.197 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:59:02.198 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:59:02.239 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:02.237+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:02.278 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:02.278 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:02.278 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:02.278 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:02.278 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:02.286 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:02.312 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:02.310+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:02.391 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:02.391 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:02.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:02.391 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:02.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135427535000005, 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:02.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135411529999999, 2024-09-17T22:59:02.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:02.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135378759999995, 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:02.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:02.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135351021999995, 2024-09-17T22:59:02.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:02.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135328603000005, 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:02.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135306322999995, 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:02.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.135098917999997, 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:02.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.134552303000007, 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:02.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:02.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.133481106999994, 2024-09-17T22:59:02.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:02.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.133469652000002, 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.133458675, 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:02.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:02.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.131384542999996, 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:02.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.131372228999993, 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:02.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.131358992000003, 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.453 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:02.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.457 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.131348162999998, 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:02.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:02.462 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.129389287999999, 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:02.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:02.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.129364929999994, 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:02.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.471 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.129335712, 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:02.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.128556042, 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:02.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.509697504, 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:02.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:02.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:02.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.488 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:02.490 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:02.490 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:02.487+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:02.590 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:02.587+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:02.634 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:02.634 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:02.643 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:02.691 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:02.689+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:02.745 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:02.745 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:02.745 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:02.745 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:02.745 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:02.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:02.792 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:02.789+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:02.858 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:02.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 118.994655999, 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:02.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:02.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.767816871999997, 2024-09-17T22:59:02.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:02.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.869 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.767798346000006, 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:02.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.767780173999995, 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.874 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:02.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.767762345999998, 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.879 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:02.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 91.767447212999997, 2024-09-17T22:59:02.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:02.887 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285622879000002, 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.888 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:02.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:02.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285613251000001, 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:02.892 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:02.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.896 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285603902999998, 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.897 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:02.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:02.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285593981000002, 2024-09-17T22:59:02.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:02.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285580525, 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:02.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:02.910 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285570712999998, 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:02.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:02.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285540344000001, 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:02.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:02.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:02.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285528021000001, 2024-09-17T22:59:02.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:02.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285518514, 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:02.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:02.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:02.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285508473, 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:02.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:02.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285499188999999, 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.935 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:02.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:02.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285489694999999, 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.939 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:02.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285478427000001, 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.285469022000001, 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:02.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:02.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:02.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:02.952 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:02.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:59:02.954 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:02.951+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:02.955 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:03.054 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:03.052+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:03.063 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:03.063 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:03.063 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:03.063 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:03.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:03.155 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:03.152+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:03.216 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:03.217 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:03.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:03.226 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:03.269 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:03.267+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:03.329 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:03.329 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1c 106/105 [PushOp(3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:head, version: 42'87, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:3d3e4e30:::rbd_data.1188cfc725d6.0000000000000002:head@42'87, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.8916053240000004, 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020122199999999999, 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.434368+0000", 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3e-07 2024-09-17T22:59:03.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435461+0000", 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010930320000000001 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435479+0000", 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8026999999999999e-05 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435490+0000", 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0681e-05 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435569+0000", 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8534000000000004e-05 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.435582+0000", 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3020000000000001e-05 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.436380+0000", 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00079829599999999999 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.7471947780000008, 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019877050000000002, 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.334 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578778+0000", 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.578779+0000", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0200000000000002e-07 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579745+0000", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096608000000000002 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579762+0000", 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7058000000000002e-05 2024-09-17T22:59:03.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579772+0000", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0505e-05 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579840+0000", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7631000000000004e-05 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.579848+0000", 2024-09-17T22:59:03.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3650000000000004e-06 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.580766+0000", 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000917464 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:03.338 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.7434825820000004, 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00207756, 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:03.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582491+0000", 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583432+0000", 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094108400000000002 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583449+0000", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6932999999999999e-05 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583460+0000", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583518+0000", 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7924000000000003e-05 2024-09-17T22:59:03.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583526+0000", 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.584568+0000", 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010420010000000001 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.342 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:33b5f7db:::rbd_data.1188aa1637fb.0000000000000000:head, version: 47'64, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:33b5f7db:::rbd_data.1188aa1637fb.0000000000000000:head@47'64, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.7102029509999994, 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018244229999999999, 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615770+0000", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.615771+0000", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616737+0000", 2024-09-17T22:59:03.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00096671699999999997 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616755+0000", 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7660999999999999e-05 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616768+0000", 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2843000000000001e-05 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616834+0000", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6478999999999998e-05 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.616843+0000", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3780000000000001e-06 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.617594+0000", 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00075164899999999996 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.346 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5738504399999993, 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:03.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:03.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a, version: 34'124, data_included: [16384~163840,212992~163840,409600~163840,606208~163840,802816~163840,999424~49152,17782901~10673,19576767~5650], data_size: 884675, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a@34'124, size: 19582417, copy_subset: [(0, 19582417)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19582417, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5691697869999999, 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021507340000000001, 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756813+0000", 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T22:59:03.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757692+0000", 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087948200000000001 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757727+0000", 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5073000000000001e-05 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757750+0000", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3201e-05 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757830+0000", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9196000000000005e-05 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757838+0000", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9409999999999996e-06 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.758954+0000", 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011154310000000001 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f, version: 42'187, data_included: [1048576~550078,2323151~47926,5670009~32074,12058624~28972,12091669~23433,15900451~27488], data_size: 709971, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f@42'187, size: 17301504, copy_subset: [(0, 17301504)], clone_subset: {}, snapset: f=[]:{f: [f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 17301504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.5446408809999994, 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001839233, 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781332+0000", 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.781333+0000", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2600000000000002e-07 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782009+0000", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000676251 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782031+0000", 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1691e-05 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782067+0000", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5867000000000001e-05 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782131+0000", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.478e-05 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.782140+0000", 2024-09-17T22:59:03.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4770000000000003e-06 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.783171+0000", 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001031541 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:03.359 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.1146804279999998, 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.364 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.1095713440000008, 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:03.365 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:03.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head, version: 47'90, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head@47'90, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0942859499999997, 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021408569999999999, 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231688+0000", 2024-09-17T22:59:03.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2500000000000005e-07 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232829+0000", 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011415430000000001 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232846+0000", 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6595e-05 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232857+0000", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0889e-05 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232947+0000", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9912000000000002e-05 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232954+0000", 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7870000000000002e-06 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.233828+0000", 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087340599999999997 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9656514420000004, 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:03.374 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.378 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [3813569~36671,4030464~65536,4530705~42381,4871284~53716,12790648~38020,15826239~34415], data_size: 270739, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15860654, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9618552099999995, 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021559650000000001, 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364339+0000", 2024-09-17T22:59:03.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022050699999999999 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364352+0000", 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3560000000000001e-05 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364363+0000", 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0764000000000001e-05 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364449+0000", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6137000000000006e-05 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364464+0000", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4712000000000001e-05 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.366274+0000", 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018097359999999999 2024-09-17T22:59:03.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8429340849999996, 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:03.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:03.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [1343488~49152,1425408~114688,1572864~114688,1720320~114688,1867776~114688,2015232~114688,2162688~114688,2310144~49152], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:03.387 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8395041859999992, 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002200498, 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:03.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487192+0000", 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072275300000000002 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487207+0000", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4586000000000001e-05 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487218+0000", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1314999999999999e-05 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487316+0000", 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8603000000000004e-05 2024-09-17T22:59:03.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487332+0000", 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5532e-05 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.488669+0000", 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013373230000000001 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.17 107/105 [PushOp(3:ed75f76e:::rbd_data.11883bd0b224.0000000000000002:21, version: 70'420, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ed75f76e:::rbd_data.11883bd0b224.0000000000000002:21@70'420, size: 1836967, copy_subset: [(0, 1836967)], clone_subset: {}, snapset: 21=[]:{21: [21]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.668093+0000", 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.657879715, 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018394970000000001, 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.668093+0000", 2024-09-17T22:59:03.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.668093+0000", 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.668094+0000", 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1099999999999995e-07 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669023+0000", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00092898199999999996 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669040+0000", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6861e-05 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669058+0000", 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8321999999999998e-05 2024-09-17T22:59:03.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669135+0000", 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6657999999999997e-05 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669148+0000", 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3069e-05 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.669933+0000", 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00078499400000000001 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.5 107/105 [PushOp(3:a2836824:::rbd_data.11883bd0b224.0000000000000008:21, version: 70'439, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a2836824:::rbd_data.11883bd0b224.0000000000000008:21@70'439, size: 1572864, copy_subset: [(0, 1572864)], clone_subset: {}, snapset: 21=[]:{21: [21]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1572864, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.845129+0000", 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.4808437439999995, 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018357239999999999, 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.396 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.845129+0000", 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.845129+0000", 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.845130+0000", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.99e-07 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846072+0000", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094247000000000005 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846089+0000", 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6316000000000001e-05 2024-09-17T22:59:03.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846099+0000", 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023999999999999e-05 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846179+0000", 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9939999999999997e-05 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846187+0000", 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9710000000000004e-06 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.846965+0000", 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00077840400000000001 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:03.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.453093076, 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:03.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:03.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.41448006300000001, 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:03.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:03.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:03.414 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4072 3.17 e116/108 3:e9e7c09d:::rbd_data.118865494b0c.0000000000000004:28 v 116'615, mlcod=109'609)", 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911914+0000", 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.41405864100000001, 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020643559999999998, 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911914+0000", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911914+0000", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911915+0000", 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:59:03.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911919+0000", 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.143e-06 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911925+0000", 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6099999999999997e-06 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911929+0000", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2139999999999998e-06 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911967+0000", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.858e-05 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911987+0000", 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9301e-05 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.913939+0000", 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019522529999999999 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.913979+0000", 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9792999999999997e-05 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.419 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198514 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.020285+0000", 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 0.30568818599999997, 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001929545, 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198514 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:03.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020285+0000", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020285+0000", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020285+0000", 2024-09-17T22:59:03.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4099999999999998e-07 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020290+0000", 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5109999999999997e-06 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020298+0000", 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8369999999999997e-06 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020302+0000", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6299999999999997e-06 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020336+0000", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3748000000000003e-05 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020419+0000", 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2929000000000005e-05 2024-09-17T22:59:03.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.020522+0000", 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010326400000000001 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.021125+0000", 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00060238399999999997 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:03.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.022156+0000", 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001031566 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.022189+0000", 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.2917999999999999e-05 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.022214+0000", 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5117e-05 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.425 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:03.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:03.426 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:03.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:03.428 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:03.424+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:03.528 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:03.526+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:03.531 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:03.531 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:03.531 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:03.531 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:03.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:03.566 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 6, 7, 5] 2024-09-17T22:59:03.567 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-17T22:59:03.567 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T22:59:03.567 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T22:59:03.585 INFO:tasks.ceph.osd.2:Started 2024-09-17T22:59:03.585 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:03.629 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:03.627+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:03.648 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:03.648 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:03.648 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:03.649 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:03.649 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:03.658 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:59:03.686 INFO:teuthology.orchestra.run.smithi159.stderr:config diff 2024-09-17T22:59:03.687 INFO:teuthology.orchestra.run.smithi159.stderr:config diff get 2024-09-17T22:59:03.687 INFO:teuthology.orchestra.run.smithi159.stderr:config get 2024-09-17T22:59:03.687 INFO:teuthology.orchestra.run.smithi159.stderr:config help [] 2024-09-17T22:59:03.687 INFO:teuthology.orchestra.run.smithi159.stderr:config set ... 2024-09-17T22:59:03.687 INFO:teuthology.orchestra.run.smithi159.stderr:admin_socket: invalid command 2024-09-17T22:59:03.688 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:59:03.689 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T22:59:03.731 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:03.729+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 87.521894149000005, 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:59:03.791 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 87.521555135, 2024-09-17T22:59:03.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:03.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.799 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.343916140000005, 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.800 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:03.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:03.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699725893, 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:03.806 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:03.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699703264, 2024-09-17T22:59:03.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:03.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:03.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:03.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699503178000001, 2024-09-17T22:59:03.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:03.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:03.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:03.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699477606000002, 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:03.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:03.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699296554999997, 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:03.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:03.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699261126000003, 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:03.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.699162225000002, 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.835 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:03.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.839 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.698986065, 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.840 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:03.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:03.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.698926569000001, 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.844 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:03.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.848 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.6886534, 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:03.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.688565224000001, 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.853 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.688512244999998, 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.858 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:03.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.688454280999999, 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.862 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:03.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:03.866 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.688169995000003, 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:03.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.868 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:03.869 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:03.870 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.687993812000002, 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.871 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:03.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:03.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.687314675000003, 2024-09-17T22:59:03.875 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:03.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:03.878 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 47.687291201000001, 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:03.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.881 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:03.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:03.883 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:03.884 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:03.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:03.886 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:03.883+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:03.910 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:59:03.986 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:03.984+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:03.999 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:03.999 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:04.000 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:04.000 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:04.009 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:04.093 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:04.091+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:04.153 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:04.153 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:04.161 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:04.188 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:04.186+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:04.199 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:04.289 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:04.286+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.310 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.269104726000002, 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.311 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:04.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.269084311, 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:04.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:04.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.269063252000002, 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:04.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:04.325 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.269040146999998, 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:04.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:04.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.269017091000002, 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.330 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.334 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268986458000001, 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:04.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:04.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:04.339 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268962481999999, 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:04.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:04.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268932304000003, 2024-09-17T22:59:04.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:04.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:04.348 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268913327, 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.349 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:04.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268474415999997, 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:04.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:04.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.268185811000002, 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.359 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:04.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:04.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:04.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.267558907000002, 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:04.365 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:04.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:04.369 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.267524467999998, 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.370 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:04.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.374 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.267136307999998, 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.375 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:04.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:04.379 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.265356175999997, 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:04.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.382 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.265341908000003, 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.384 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:04.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.265331451999998, 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:04.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.393 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.265313171000003, 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:04.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.264893702999998, 2024-09-17T22:59:04.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:04.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 48.264267457999999, 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:04.403 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:04.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:04.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:04.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:04.408 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:04.410 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:04.410 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:04.410 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:04.411 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:04.411 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:04.411 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:59:04.411 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:04.412 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:04.412 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:04.413 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:04.414 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:04.407+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:04.423 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:04.424 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:04.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:59:04.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:59:04.510 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:04.508+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:04.526 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:04.526 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:04.535 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:04.612 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:04.609+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:04.637 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:04.638 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:04.646 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:04.713 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:04.710+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:04.749 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:04.750 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:04.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:04.801 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:04.798+0000 7f0adca0b640 -1 Falling back to public interface 2024-09-17T22:59:04.813 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:04.811+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:04.874 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:04.874 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:04.883 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:04.924 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:04.922+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:05.014 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:05.012+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:05.039 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.039 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:05.133 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:05.131+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:05.194 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.194 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.202 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:05.243 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:05.241+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:05.318 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:05.315+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:05.346 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.346 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.355 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:05.418 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:05.416+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:05.606 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.606 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.607 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:05.607 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:05.604+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:05.707 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:05.704+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:05.751 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.752 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.760 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:05.807 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:05.805+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:05.864 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:05.864 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:05.873 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:05.915 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:05.912+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:05.928 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:59:06.010 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:06.007+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:06.035 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:06.035 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:06.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:06.110 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:06.108+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:06.149 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:06.149 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:06.157 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:06.211 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:06.209+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:06.262 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:06.262 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:06.271 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:06.312 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:06.310+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:06.413 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:06.411+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:06.415 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:06.415 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:06.424 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:06.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:06.514 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:06.511+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:06.548 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:06.548 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:06.564 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:06.565 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:06.565 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:59:06.568 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:06.568 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:06.577 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:06.580 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:06.580 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:06.597 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:06.597 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:06.614 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:06.614 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:06.615 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:06.612+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:06.630 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:06.631 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:06.647 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:06.648 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:06.651 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:59:06.651 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:59:06.682 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:06.683 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:06.683 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:06.683 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:06.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:06.715 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:06.713+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:06.816 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:06.814+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:06.834 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:06.835 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:06.835 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:06.844 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:06.917 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:06.915+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:06.929 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:59:06.964 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:06.964 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:06.964 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:06.964 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:06.964 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 90.695426507999997, 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.965 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:59:06.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:06.969 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 90.695087494000006, 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.970 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:06.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:06.978 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.517448498999997, 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.979 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:06.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:06.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:06.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.873258251999999, 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:06.986 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:06.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:06.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:06.991 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.873235622999999, 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.992 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:06.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:06.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.873035537, 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:06.998 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:06.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:07.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.873009965000001, 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:07.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:07.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:07.008 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.872828914000003, 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:07.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:07.013 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.872793485000003, 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.872694584000001, 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.020 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:07.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:07.025 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.872518423999999, 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:07.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.030 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.872458928, 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.031 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:07.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:07.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.862185758999999, 2024-09-17T22:59:07.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:07.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:07.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.862097583000001, 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:07.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:07.047 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.862044603999998, 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.048 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:07.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:07.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.053 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.861986639999998, 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.054 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:07.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.861702354000002, 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.861526171000001, 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.065 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:07.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.860847034000003, 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.070 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:07.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:07.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.86082356, 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.076 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:07.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.081 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.081 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.081 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:07.083 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:07.080+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:07.182 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:07.180+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:07.227 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.227 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:07.227 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:07.227 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.236 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:07.283 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:07.281+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:07.380 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:07.380 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:07.380 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.384 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:07.382+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:07.388 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:07.485 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:07.483+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451899767, 2024-09-17T22:59:07.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:07.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.497 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451879351999999, 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.498 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:07.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.500 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451858293000001, 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:07.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451835187999997, 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.511 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451812132000001, 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.512 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:07.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:07.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451781498999999, 2024-09-17T22:59:07.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:07.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451757522999998, 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:07.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:07.525 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451727345000002, 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:07.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451708367999998, 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:07.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.451269457000002, 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.535 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:07.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.450980852000001, 2024-09-17T22:59:07.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:07.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.543 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.450353948, 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.544 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.548 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.450319509000003, 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:07.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.449931349000003, 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.553 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.555 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:07.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.448151217000003, 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.448136949000002, 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.562 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:07.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.566 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.448126492999997, 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:07.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:07.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.448108212000001, 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.571 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:07.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.447688743999997, 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.576 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 51.447062498999998, 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.581 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.585 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.587 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:07.587 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:07.584+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:07.687 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:07.685+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:07.689 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:07.690 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.698 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:07.788 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:07.786+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:07.800 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:07.801 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:07.809 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:07.889 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:07.886+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.922 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 68.765564298000001, 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.923 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:59:07.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:59:07.927 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.928 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.929 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413775899000001, 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:59:07.931 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.932 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.933 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413755189, 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:59:07.934 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.935 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:59:07.936 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.937 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.938 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413735394, 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:59:07.939 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:07.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.941 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413716151000003, 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.943 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:59:07.945 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:59:07.946 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413695011000002, 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:59:07.947 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.948 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:59:07.949 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.950 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.951 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413669921999997, 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.952 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.953 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.954 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:59:07.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413650578999999, 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:59:07.956 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.957 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:59:07.958 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:59:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413628555999999, 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:59:07.960 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.961 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:59:07.962 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:59:07.963 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413534415000001, 2024-09-17T22:59:07.964 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.965 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413507643000003, 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.969 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:59:07.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:59:07.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413487117000003, 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:59:07.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:59:07.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.976 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413465823999999, 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.977 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:59:07.980 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413436949000001, 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.981 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:07.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413395870000002, 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:59:07.985 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.986 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:59:07.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 54.413371847000001, 2024-09-17T22:59:07.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.992 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.993 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.528629219999999, 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:59:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.997 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.528607592, 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:07.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:07.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:59:08.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:08.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.528586408000002, 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.003 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:08.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 50.528565327999999, 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:08.007 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:08.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:59:08.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:08.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:08.011 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:08.013 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:08.013 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:08.010+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:08.114 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:08.111+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:08.125 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.125 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:08.125 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:08.125 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.133 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:08.215 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:08.212+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:08.238 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.238 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:08.238 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:08.238 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:08.238 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:08.316 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:08.314+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:08.352 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.352 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:08.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.964121431999999, 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.353 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:08.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.925148479999997, 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.358 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:08.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.925127703999998, 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.362 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:08.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.366 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.925108913999999, 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:08.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.925063702999999, 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:08.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.925029875, 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:08.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.220112608000001, 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:08.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.220092139999998, 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:08.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:08.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:08.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.220070579000001, 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:08.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.220050762, 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.393 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:08.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:08.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.220030774000001, 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.398 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:08.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:08.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189941 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219468916, 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060996437000000001, 2024-09-17T22:59:08.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189941 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129927+0000", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129931+0000", 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1409999999999998e-06 2024-09-17T22:59:08.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129947+0000", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6359999999999999e-05 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129955+0000", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1790000000000002e-06 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190765+0000", 2024-09-17T22:59:08.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060810757999999999 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190831+0000", 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5188e-05 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190923+0000", 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2139999999999995e-05 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219365361000001, 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:08.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219342615999999, 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:08.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219321483000002, 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:08.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:08.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219301366, 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219274156000001, 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219103295, 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:08.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:08.433 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.219078976999999, 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:08.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.213804974999999, 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:08.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:08.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.446 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:08.448 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:08.445+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:08.548 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:08.546+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:08.592 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:08.592 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:08.592 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.600 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:08.628 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:08.625+0000 7f0adca0b640 -1 osd.2 74 log_to_monitors true 2024-09-17T22:59:08.649 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:08.647+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:08.652 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:08.690 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:08.703 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:08.703 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:08.703 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:08.703 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.712 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:08.753 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:08.751+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:08.788 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:08.788 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:08.820 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:08.820 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T22:59:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:no valid command found; 10 closest matches: 2024-09-17T22:59:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:0 2024-09-17T22:59:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:1 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:2 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:abort 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:assert 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs files list 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:bluefs stats 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator dump block 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:bluestore allocator fragmentation block 2024-09-17T22:59:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:admin_socket: invalid command 2024-09-17T22:59:08.823 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:59:08.823 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T22:59:08.837 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:08.837 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:08.853 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:08.851+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:08.853 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:08.854 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:08.857 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.602006750000001, 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601990744999995, 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601957975000005, 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:08.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601930237000005, 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.872 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:08.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:08.876 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601907818000001, 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601885538000005, 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:08.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601678132999993, 2024-09-17T22:59:08.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:08.890 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.601131518000003, 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.891 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:08.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.600060322000004, 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.600048866999998, 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:08.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.600037889999996, 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:08.905 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:08.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:08.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.597963758000006, 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.914 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.597951444000003, 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:08.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:08.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.597938206999999, 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:08.919 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:08.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.923 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.597927377999994, 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.924 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:08.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.928 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.595968502999995, 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:08.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.595944145000004, 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.933 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:08.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.595914926999995, 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:08.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:08.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:08.942 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 92.595135256999995, 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:08.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:08.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.976276718999999, 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:08.947 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:08.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:08.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:08.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:08.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:08.955 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:08.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:59:08.956 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:08.957 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:08.957 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:08.957 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:08.958 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:08.958 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:08.959 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:59:08.959 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:08.959 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:59:08.960 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:08.953+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:08.976 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:08.974+0000 7f0ad00b7640 -1 osd.2 122 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:59:09.056 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:09.054+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:09.073 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.073 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.082 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:09.157 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:09.155+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:09.186 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.186 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.195 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:09.261 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:09.259+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:09.339 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.339 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.348 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:09.392 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:09.390+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:09.451 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.451 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:09.501 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:09.499+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:09.564 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:09.562+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:09.603 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.604 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.612 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:09.665 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:09.662+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:09.754 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.754 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.763 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:09.766 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:09.764+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:09.867 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:09.865+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:09.868 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.868 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:09.877 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:09.953 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:59:09.968 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:09.966+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:09.993 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:09.993 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:10.087 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:10.085+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:10.147 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.147 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.155 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:10.170 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:10.168+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:10.271 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:10.269+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.309 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:10.392 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:10.390+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:10.452 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.452 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.460 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:10.473 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:10.471+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:10.574 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:10.572+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:10.603 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.603 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.611 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:10.674 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:10.672+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:10.717 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.717 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.725 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:10.775 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:10.773+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:10.828 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.828 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.836 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:10.880 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:10.878+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:10.939 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:10.939 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:10.948 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:10.960 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:10.993 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:10.991+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:11.062 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:11.062 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:11.071 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:11.112 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:11.110+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:11.120 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:11.121 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:11.137 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:11.138 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:11.153 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:11.154 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T22:59:11.169 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:11.169 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:11.178 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:11.176+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:11.186 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:11.186 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:11.202 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:11.202 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:11.216 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:11.217 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:11.217 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:11.217 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:11.217 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:11.219 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:11.219 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:11.226 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:11.235 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:11.235 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:11.311 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:11.309+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:11.372 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.983362059999997, 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:11.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.944389108000003, 2024-09-17T22:59:11.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:11.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:11.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.944368332000003, 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:11.382 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:11.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.386 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.944349541999998, 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.387 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:11.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.944304330999998, 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.392 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:11.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 53.944270502999998, 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.400 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:11.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:11.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:11.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:11.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:11.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:11.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:11.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.410 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.239353235999999, 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:11.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.239332768000001, 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:11.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:11.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:11.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.422 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.239311207, 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.427 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.239291389999998, 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.428 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:11.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:11.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.434 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.239271402, 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.435 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:11.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189941 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238709543999999, 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060996437000000001, 2024-09-17T22:59:11.442 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189941 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129926+0000", 2024-09-17T22:59:11.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129927+0000", 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7100000000000001e-07 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129931+0000", 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1409999999999998e-06 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129947+0000", 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6359999999999999e-05 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129955+0000", 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1790000000000002e-06 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190765+0000", 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060810757999999999 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190831+0000", 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5188e-05 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190923+0000", 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2139999999999995e-05 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238605989, 2024-09-17T22:59:11.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:11.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.453 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238583244000001, 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.454 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:11.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:11.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238562111, 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:11.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238541993999998, 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:11.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:11.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238514783999999, 2024-09-17T22:59:11.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:11.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:11.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238343922999999, 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.474 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:11.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:11.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.238319605000001, 2024-09-17T22:59:11.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:11.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:11.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.233045603000001, 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:11.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:11.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:11.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.490 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:11.491 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:11.492 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:11.489+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:11.592 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:11.590+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:11.636 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:11.636 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:11.636 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:11.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:11.694 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:11.691+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:11.748 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:11.749 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:11.749 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:11.757 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:11.795 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:11.792+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:11.896 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:11.894+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:11.903 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647347027999999, 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:11.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:11.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647331023000007, 2024-09-17T22:59:11.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:11.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647298253000002, 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:11.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:11.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647270515000002, 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:11.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:11.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.924 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647248095999998, 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:11.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647225816000002, 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:11.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.647018411000005, 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:11.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.646471796, 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:11.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:11.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.645400600000002, 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:11.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:11.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.645389144999996, 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:11.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.645378167999993, 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:11.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.643304036000004, 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:11.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.643291722000001, 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.964 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:11.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.968 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.643278484999996, 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.969 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:11.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.643267656000006, 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:11.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.641308781000006, 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:11.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:11.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:11.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.641284423000002, 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:11.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.641255204999993, 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.988 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:11.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 95.640475534999993, 2024-09-17T22:59:11.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:11.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:11.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:11.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 14.021616997000001, 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:11.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:11.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:11.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:12.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:12.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.004 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.005 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:59:12.006 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:12.004+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:12.007 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:12.107 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:12.105+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:12.155 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:12.155 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:12.156 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.165 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:12.208 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:12.205+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:12.308 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:12.306+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:12.312 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.312 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:12.313 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:12.313 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:12.313 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.322 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:12.409 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:12.407+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.428 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.56451165499999, 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:12.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:12.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:12.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:12.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.337672528, 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:12.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.33765400199999, 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:12.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.33763583, 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.448 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:12.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:12.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.337618002, 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.453 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:12.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:12.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 101.337302869, 2024-09-17T22:59:12.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:12.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:12.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:12.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855478535000003, 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:12.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:12.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855468907000002, 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:12.469 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:12.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855459559000003, 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:12.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:12.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855449637, 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.479 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:12.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.483 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855436181000002, 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:12.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855426369, 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.492 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855395999999999, 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:12.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855383676999999, 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:12.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855374169999997, 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.502 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855364129000002, 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:12.506 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:12.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855354845000001, 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:12.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.515 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855345350999997, 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.516 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:12.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855334083000002, 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.520 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:12.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.524 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.855324678000002, 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.525 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:12.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:12.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:12.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:12.529 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.532 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.532 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.532 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:12.534 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:12.532+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:12.635 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:12.632+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:12.682 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:12.682 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:12.682 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.691 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:12.738 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:12.735+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:12.793 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:12.794 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:12.794 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:12.794 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:12.802 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:12.843 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:12.841+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:12.941 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:12.939+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:12.946 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:12.946 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.360768319000002, 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00207756, 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582491+0000", 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583432+0000", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094108400000000002 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583449+0000", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6932999999999999e-05 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583460+0000", 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583518+0000", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7924000000000003e-05 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583526+0000", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.584568+0000", 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010420010000000001 2024-09-17T22:59:12.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.191136177000001, 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:12.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a, version: 34'124, data_included: [16384~163840,212992~163840,409600~163840,606208~163840,802816~163840,999424~49152,17782901~10673,19576767~5650], data_size: 884675, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a@34'124, size: 19582417, copy_subset: [(0, 19582417)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19582417, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.186455523999999, 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021507340000000001, 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756813+0000", 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757692+0000", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087948200000000001 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757727+0000", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5073000000000001e-05 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757750+0000", 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3201e-05 2024-09-17T22:59:12.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757830+0000", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9196000000000005e-05 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757838+0000", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9409999999999996e-06 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.758954+0000", 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011154310000000001 2024-09-17T22:59:12.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.731966164999999, 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:12.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.726857080999999, 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:12.964 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:12.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head, version: 47'90, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head@47'90, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.711571686999999, 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021408569999999999, 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231688+0000", 2024-09-17T22:59:12.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2500000000000005e-07 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232829+0000", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011415430000000001 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232846+0000", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6595e-05 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232857+0000", 2024-09-17T22:59:12.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0889e-05 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232947+0000", 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9912000000000002e-05 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232954+0000", 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7870000000000002e-06 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.233828+0000", 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087340599999999997 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.582937178999998, 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [3813569~36671,4030464~65536,4530705~42381,4871284~53716,12790648~38020,15826239~34415], data_size: 270739, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15860654, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.579140946999999, 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021559650000000001, 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.977 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364339+0000", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022050699999999999 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364352+0000", 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3560000000000001e-05 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364363+0000", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0764000000000001e-05 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364449+0000", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6137000000000006e-05 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364464+0000", 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4712000000000001e-05 2024-09-17T22:59:12.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.366274+0000", 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018097359999999999 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.460219821999999, 2024-09-17T22:59:12.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:12.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [1343488~49152,1425408~114688,1572864~114688,1720320~114688,1867776~114688,2015232~114688,2162688~114688,2310144~49152], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:12.985 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.456789922999999, 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002200498, 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:12.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487192+0000", 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072275300000000002 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487207+0000", 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4586000000000001e-05 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487218+0000", 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1314999999999999e-05 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487316+0000", 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8603000000000004e-05 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487332+0000", 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5532e-05 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.488669+0000", 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013373230000000001 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.989 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 15.070378813, 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:12.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:12.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 10.031765800000001, 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:12.996 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:12.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:12.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:13.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:13.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0299998719999994, 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:13.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:13.009 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0245635740000001, 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:13.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:13.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:13.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1509 3.10 e120/114 3:0d15f2c1:::rbd_data.1188f4e2c696.000000000000001b:24 v 120'638, mlcod=119'633)", 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 6.0049005590000002, 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020874230000000001, 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938359+0000", 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4199999999999996e-07 2024-09-17T22:59:13.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938363+0000", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5260000000000004e-06 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938372+0000", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4819999999999994e-06 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938378+0000", 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0020000000000003e-06 2024-09-17T22:59:13.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938880+0000", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050230700000000001 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938896+0000", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5971e-05 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.940410+0000", 2024-09-17T22:59:13.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015144939999999999 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.940446+0000", 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5098999999999997e-05 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.9998630889999998, 2024-09-17T22:59:13.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:13.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.9956640310000004, 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:13.025 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:13.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.030 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.9999249949999998, 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:13.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:13.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:199116 3.17 e121/108 3:ed33c955:::rbd_header.1188b9dbd70:head v 121'628, mlcod=120'626)", 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.999271351, 2024-09-17T22:59:13.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00221332, 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943988+0000", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943995+0000", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1239999999999999e-06 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944093+0000", 2024-09-17T22:59:13.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7671999999999996e-05 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944104+0000", 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944212+0000", 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108087 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944237+0000", 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4841000000000002e-05 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946157+0000", 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019206939999999999 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946201+0000", 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3225e-05 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.039 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 4.8765618210000001, 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:13.040 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:13.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:13.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:13.044 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:13.046 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:13.047 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:13.043+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:13.047 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:13.149 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:13.147+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:13.191 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:13.191 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:13.192 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:13.192 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:13.200 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:13.246 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:13.250 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:13.248+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:13.346 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:13.346 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:13.347 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:13.347 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:13.347 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:13.353 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:13.351+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:13.356 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:13.406 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:13.407 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:13.424 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:13.441 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:13.441 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:13.454 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:13.452+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:13.457 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:13.474 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:13.474 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:13.491 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:13.492 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:13.503 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:13.503 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:13.508 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:13.508 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:13.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:13.525 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:13.555 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:13.553+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:13.629 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:13.629 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:13.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:13.683 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:13.680+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:13.741 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:13.742 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:13.750 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:13.757 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:13.755+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:13.824 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:13.858 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:13.856+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:13.896 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:13.896 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:13.905 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:13.942 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:13.942 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:13.951 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T22:59:13.993 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:13.990+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:14.025 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.026 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.036 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:14.060 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:14.058+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:14.161 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:14.158+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:14.184 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.184 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.193 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:14.262 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:14.260+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:14.337 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.337 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:14.363 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:14.360+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:14.451 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.451 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:14.504 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:14.502+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:14.563 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.563 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.564 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:14.562+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:14.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:14.665 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:14.663+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:14.716 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.717 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.726 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:14.766 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:14.763+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:14.866 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:14.864+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:14.869 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:14.869 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:14.879 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:14.967 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:14.965+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T22:59:15.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T22:59:15.027 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:15.027 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:15.036 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:15.047 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:59:15.077 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:15.075+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:15.169 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:15.166+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:15.181 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:15.181 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:15.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:15.269 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:15.267+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:15.334 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:15.334 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:15.342 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:15.386 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:15.384+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:15.447 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:15.447 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:15.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:15.497 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:15.495+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:15.537 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:15.572 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:15.570+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:15.600 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:15.600 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:15.609 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:15.656 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:15.656 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:15.673 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:15.673 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:15.674 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:15.671+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:15.689 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:15.689 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:15.704 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:15.705 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:15.721 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:15.721 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:15.737 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:15.737 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:15.754 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:15.754 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:15.754 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:15.754 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:15.754 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:15.755 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:15.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:15.770 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:15.770 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:15.774 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:15.772+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:15.866 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:15.866 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:15.866 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:15.866 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:15.866 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:15.875 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:15.876 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:15.874+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:15.977 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:15.975+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:15.981 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:15.981 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.c 106/105 [PushOp(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9, version: 33'19, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34500d47:::rbd_data.118846a2cbc0.0000000000000002:9@33'19, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.395283724999999, 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00207756, 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:15.982 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582490+0000", 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:15.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.582491+0000", 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1699999999999998e-07 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583432+0000", 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00094108400000000002 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583449+0000", 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6932999999999999e-05 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583460+0000", 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583518+0000", 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7924000000000003e-05 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:15.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.583526+0000", 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.584568+0000", 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010420010000000001 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.225651583000001, 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:15.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:15.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a, version: 34'124, data_included: [16384~163840,212992~163840,409600~163840,606208~163840,802816~163840,999424~49152,17782901~10673,19576767~5650], data_size: 884675, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fe911258:::rbd_data.1188b0ea165c.0000000000000003:a@34'124, size: 19582417, copy_subset: [(0, 19582417)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19582417, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.22097093, 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021507340000000001, 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:15.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756803+0000", 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.756813+0000", 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4099999999999997e-06 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757692+0000", 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087948200000000001 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757727+0000", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5073000000000001e-05 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757750+0000", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3201e-05 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757830+0000", 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9196000000000005e-05 2024-09-17T22:59:15.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.757838+0000", 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9409999999999996e-06 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.758954+0000", 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011154310000000001 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:15.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.766481571, 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:15.997 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:15.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:15.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:16.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:16.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:16.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:16.007 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:16.008 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.761372486999999, 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:16.009 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:16.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:16.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:16.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:16.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.020 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head, version: 47'90, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6ecf453b:::rbd_data.1188aa1637fb.0000000000000003:head@47'90, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.746087093, 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021408569999999999, 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.021 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231687+0000", 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.231688+0000", 2024-09-17T22:59:16.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2500000000000005e-07 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232829+0000", 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011415430000000001 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232846+0000", 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6595e-05 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232857+0000", 2024-09-17T22:59:16.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0889e-05 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232947+0000", 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9912000000000002e-05 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.232954+0000", 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7870000000000002e-06 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.233828+0000", 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087340599999999997 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.617452584999999, 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:16.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:16.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:16.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [3813569~36671,4030464~65536,4530705~42381,4871284~53716,12790648~38020,15826239~34415], data_size: 270739, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15860654, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.613656353, 2024-09-17T22:59:16.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021559650000000001, 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:16.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364339+0000", 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022050699999999999 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364352+0000", 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3560000000000001e-05 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364363+0000", 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0764000000000001e-05 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364449+0000", 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6137000000000006e-05 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364464+0000", 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4712000000000001e-05 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.366274+0000", 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018097359999999999 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.037 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.494735228, 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:16.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [1343488~49152,1425408~114688,1572864~114688,1720320~114688,1867776~114688,2015232~114688,2162688~114688,2310144~49152], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 21.491305328999999, 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002200498, 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.042 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487192+0000", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072275300000000002 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487207+0000", 2024-09-17T22:59:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4586000000000001e-05 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487218+0000", 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1314999999999999e-05 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487316+0000", 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8603000000000004e-05 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487332+0000", 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5532e-05 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.046 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.488669+0000", 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013373230000000001 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 18.104894218999998, 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.048 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.049 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:16.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.055 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:16.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.058 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.066281205999999, 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.059 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:16.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:16.063 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:16.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:16.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.064515278, 2024-09-17T22:59:16.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.070 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:16.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:16.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 12.059078980000001, 2024-09-17T22:59:16.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:16.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:16.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:16.082 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1509 3.10 e120/114 3:0d15f2c1:::rbd_data.1188f4e2c696.000000000000001b:24 v 120'638, mlcod=119'633)", 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0394159649999999, 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0020874230000000001, 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.083 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938358+0000", 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938359+0000", 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4199999999999996e-07 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938363+0000", 2024-09-17T22:59:16.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5260000000000004e-06 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938372+0000", 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4819999999999994e-06 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938378+0000", 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0020000000000003e-06 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938880+0000", 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00050230700000000001 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.938896+0000", 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5971e-05 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.940410+0000", 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0015144939999999999 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.940446+0000", 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5098999999999997e-05 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:16.088 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0343784950000003, 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:16.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:16.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:16.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:16.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.0301794369999993, 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:16.094 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:16.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:16.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:16.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.101 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.0344404009999995, 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:16.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:16.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:16.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:199116 3.17 e121/108 3:ed33c955:::rbd_header.1188b9dbd70:head v 121'628, mlcod=120'626)", 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:16.106 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.0337867569999997, 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00221332, 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943988+0000", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943995+0000", 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1239999999999999e-06 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944093+0000", 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7671999999999996e-05 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944104+0000", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944212+0000", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108087 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944237+0000", 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4841000000000002e-05 2024-09-17T22:59:16.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946157+0000", 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019206939999999999 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946201+0000", 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3225e-05 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 7.9110772269999998, 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:16.112 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:16.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:16.116 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:16.117 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:59:16.118 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T22:59:16.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:16.119 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:16.116+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:16.120 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T22:59:16.219 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:16.218+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:16.266 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:16.266 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:16.276 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:16.320 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:16.318+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:16.380 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:16.380 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:16.380 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:16.380 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:16.380 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:16.389 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:16.430 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:16.428+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:16.521 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:16.519+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26, version: 76'382, data_included: [0~631961,1179648~65536,4194304~131072], data_size: 828569, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26@76'382, size: 7340032, copy_subset: [(0, 7340032)], clone_subset: {}, snapset: 26=[]:{26: [26]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7340032, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.534 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5757782010000003, 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021080690000000002, 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954950+0000", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955691+0000", 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074070899999999999 2024-09-17T22:59:16.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955710+0000", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9463999999999999e-05 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955731+0000", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0437000000000001e-05 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955812+0000", 2024-09-17T22:59:16.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1221e-05 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955822+0000", 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0016e-05 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.957057+0000", 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012356509999999999 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.538 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:head, version: 76'383, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:head@76'383, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.958308+0000", 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5724199580000002, 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00077591699999999999, 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.539 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958308+0000", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958308+0000", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958308+0000", 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:59:16.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958318+0000", 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5829999999999996e-06 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958333+0000", 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5075999999999999e-05 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.541 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958344+0000", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1069e-05 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958421+0000", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7373999999999995e-05 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.958434+0000", 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.225e-05 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.542 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.959084+0000", 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00064992499999999998 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5461969299999998, 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001379094, 2024-09-17T22:59:16.543 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.544 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3799999999999997e-07 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985075+0000", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054357699999999995 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985092+0000", 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7041000000000002e-05 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.545 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985103+0000", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0740999999999999e-05 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985162+0000", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9290999999999999e-05 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985170+0000", 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7579999999999992e-06 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.546 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985910+0000", 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074014800000000004 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5438551770000002, 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013103189999999999, 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.547 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986873+0000", 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:59:16.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987326+0000", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00045334100000000001 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987343+0000", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6495e-05 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987354+0000", 2024-09-17T22:59:16.549 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0791999999999999e-05 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987411+0000", 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7445000000000003e-05 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987419+0000", 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0139999999999998e-06 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.550 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.988183+0000", 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00076374600000000004 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35, version: 91'439, data_included: [12288~28672,102400~28672,143360~28672,233472~28672,274432~28672,364544~28672,405504~28672,495616~28672,536576~28672,626688~28672,667648~28672,757760~28672,798720~28672,888832~28672,929792~20480,1019904~28672], data_size: 450560, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35@91'439, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 35=[]:{35: [35]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5270893980000002, 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017163370000000001, 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.551 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003639+0000", 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.71e-07 2024-09-17T22:59:16.552 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003943+0000", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00030434599999999999 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003969+0000", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5700000000000001e-05 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003994+0000", 2024-09-17T22:59:16.553 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4416000000000001e-05 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004076+0000", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2137000000000003e-05 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004086+0000", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0271999999999999e-05 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.005355+0000", 2024-09-17T22:59:16.554 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001268595 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:263d8630:::rbd_data.11881ee9ab63.000000000000006a:36, version: 92'462, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:263d8630:::rbd_data.11881ee9ab63.000000000000006a:36@92'462, size: 262144, copy_subset: [(0, 262144)], clone_subset: {}, snapset: 36=[]:{36: [36]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262144, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.018049+0000", 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5126782179999996, 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00071438700000000003, 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.555 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018049+0000", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018049+0000", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018050+0000", 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1799999999999995e-07 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.556 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018217+0000", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00016752 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018239+0000", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1457000000000001e-05 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018259+0000", 2024-09-17T22:59:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9734000000000001e-05 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018316+0000", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7317e-05 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018323+0000", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2679999999999997e-06 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.018764+0000", 2024-09-17T22:59:16.558 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00044047299999999998 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1528 3.4 e124/123 3:2363738a:::rbd_data.1188b9dbd70.0000000000000032:23 v 124'482, mlcod=115'479)", 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4765997750000004, 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022419990000000002, 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.559 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.560 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054135+0000", 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2750000000000001e-06 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054146+0000", 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0829000000000001e-05 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054153+0000", 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0099999999999998e-06 2024-09-17T22:59:16.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054191+0000", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8550999999999997e-05 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054205+0000", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3891e-05 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056337+0000", 2024-09-17T22:59:16.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021323190000000001 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056370+0000", 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2455e-05 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1529 3.4 e124/123 3:220bdc83:::rbd_data.1188f4e2c696.000000000000010a:24 v 124'484, mlcod=115'480)", 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4735215410000002, 2024-09-17T22:59:16.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025955399999999999, 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057207+0000", 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7899999999999998e-07 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057212+0000", 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0479999999999998e-06 2024-09-17T22:59:16.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057221+0000", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057227+0000", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3369999999999996e-06 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057277+0000", 2024-09-17T22:59:16.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9292000000000002e-05 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057290+0000", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059759+0000", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0024690379999999998 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059802+0000", 2024-09-17T22:59:16.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2106e-05 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1530 3.4 e124/123 3:222138c9:::rbd_data.1188f4e2c696.000000000000003e:24 v 124'486, mlcod=115'480)", 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4730320690000003, 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023558860000000002, 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057703+0000", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8859999999999998e-06 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057711+0000", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2949999999999994e-06 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057717+0000", 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4779999999999999e-06 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057755+0000", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8606000000000003e-05 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057767+0000", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1484999999999999e-05 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060021+0000", 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022539080000000002 2024-09-17T22:59:16.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060051+0000", 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0679000000000002e-05 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1531 3.4 e124/123 3:2668e2f4:::rbd_data.1188f4e2c696.0000000000000042:24 v 124'488, mlcod=124'484)", 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.470289889, 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001073601, 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.572 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060444+0000", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.237e-06 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060454+0000", 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0824e-05 2024-09-17T22:59:16.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060461+0000", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060501+0000", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0577000000000001e-05 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060515+0000", 2024-09-17T22:59:16.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3429e-05 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061462+0000", 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00094694600000000005 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061511+0000", 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9435999999999999e-05 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1532 3.4 e124/123 3:21ded978:::rbd_data.1188f4e2c696.000000000000004b:24 v 124'490, mlcod=124'484)", 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.060977+0000", 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4697507119999997, 2024-09-17T22:59:16.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00081218100000000004, 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060977+0000", 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060977+0000", 2024-09-17T22:59:16.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060978+0000", 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5199999999999996e-07 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060984+0000", 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3679999999999998e-06 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.579 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060994+0000", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0591000000000001e-05 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061002+0000", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1450000000000002e-06 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061055+0000", 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3560000000000002e-05 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061073+0000", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8087000000000001e-05 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061752+0000", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00067886599999999997 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061789+0000", 2024-09-17T22:59:16.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6912000000000002e-05 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1533 3.4 e124/123 3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26 v 124'492, mlcod=124'488)", 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.063567+0000", 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4671607409999998, 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00071652800000000002, 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.582 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063567+0000", 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063567+0000", 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063568+0000", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7100000000000001e-07 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063572+0000", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5360000000000003e-06 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063580+0000", 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0879999999999995e-06 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063586+0000", 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9889999999999997e-06 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063624+0000", 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7923000000000003e-05 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.063636+0000", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2289999999999999e-05 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.064252+0000", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061561099999999996 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.064283+0000", 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1420000000000001e-05 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.586 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1534 3.4 e124/123 3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30 v 124'494, mlcod=124'490)", 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4603668699999997, 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001260253, 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.68e-07 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070367+0000", 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8780000000000003e-06 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070377+0000", 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7440000000000002e-06 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.589 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070384+0000", 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9140000000000002e-06 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070425+0000", 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0800999999999998e-05 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070446+0000", 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1603e-05 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071578+0000", 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001131878 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071621+0000", 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2766999999999999e-05 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.591 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1536 3.4 e124/123 3:26e14644:::rbd_data.1188b7e16ab4.0000000000000005:34 v 124'498, mlcod=124'494)", 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.075315+0000", 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4554126780000001, 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00079615899999999995, 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075315+0000", 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075315+0000", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075316+0000", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9999999999999996e-07 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075322+0000", 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2190000000000001e-06 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075334+0000", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1792e-05 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075340+0000", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6449999999999999e-06 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075381+0000", 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1010000000000002e-05 2024-09-17T22:59:16.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.075403+0000", 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1429000000000001e-05 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.076084+0000", 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00068115000000000005 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.076111+0000", 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7114000000000002e-05 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1537 3.4 e124/123 3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35 v 124'500, mlcod=124'496)", 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.077030+0000", 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4536979519999997, 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0009922799999999999, 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.596 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077030+0000", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077030+0000", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077030+0000", 2024-09-17T22:59:16.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8599999999999998e-07 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077038+0000", 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2509999999999997e-06 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077049+0000", 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1646e-05 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077056+0000", 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8469999999999998e-06 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077099+0000", 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3420000000000001e-05 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077120+0000", 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.018e-05 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.077980+0000", 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00085997099999999996 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078022+0000", 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2279e-05 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.600 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1538 3.4 e124/123 3:22e62e78:::rbd_data.11881ee9ab63.000000000000005d:36 v 124'502, mlcod=124'498)", 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.078934+0000", 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4517940649999996, 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098314800000000009, 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078934+0000", 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078934+0000", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078934+0000", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6199999999999997e-07 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078940+0000", 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1600000000000003e-06 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078951+0000", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0879999999999999e-05 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.078959+0000", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8390000000000007e-06 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079002+0000", 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2970999999999999e-05 2024-09-17T22:59:16.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079028+0000", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5965999999999998e-05 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079876+0000", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00084799699999999997 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079917+0000", 2024-09-17T22:59:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0673000000000002e-05 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1539 3.4 e124/123 3:263d8630:::rbd_data.11881ee9ab63.000000000000006a:36 v 124'504, mlcod=124'498)", 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4512817770000002, 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001105275, 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.605 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079447+0000", 2024-09-17T22:59:16.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079453+0000", 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079461+0000", 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1699999999999997e-06 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.607 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079468+0000", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5570000000000001e-06 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079506+0000", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7540999999999999e-05 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079530+0000", 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4508999999999999e-05 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080522+0000", 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099224300000000007 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080551+0000", 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8736999999999999e-05 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1541 3.4 e124/123 3:2750c8ed:::rbd_data.11881ee9ab63.000000000000005c:36 v 124'508, mlcod=124'502)", 2024-09-17T22:59:16.609 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.081674+0000", 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.4490538480000001, 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00084117999999999997, 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081674+0000", 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081674+0000", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081674+0000", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.68e-07 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081679+0000", 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3710000000000002e-06 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081686+0000", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8189999999999992e-06 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081691+0000", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5140000000000003e-06 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081726+0000", 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4527000000000003e-05 2024-09-17T22:59:16.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.081737+0000", 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1066999999999999e-05 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.082482+0000", 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074576999999999998 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.082515+0000", 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2543999999999999e-05 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.0 125/124 [PushOp(3:00f9cb00:::rbd_data.1188b7e16ab4.0000000000000000:34, version: 90'314, data_included: [0~131072,3623654~27834], data_size: 158906, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:00f9cb00:::rbd_data.1188b7e16ab4.0000000000000000:34@90'314, size: 4980736, copy_subset: [(0, 4980736)], clone_subset: {}, snapset: 34=[]:{34: [34]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4980736, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:12.038032+0000", 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.4926957920000001, 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00082829900000000001, 2024-09-17T22:59:16.614 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038032+0000", 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038032+0000", 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038032+0000", 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5300000000000004e-07 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038083+0000", 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0649999999999998e-05 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038099+0000", 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5848000000000001e-05 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038109+0000", 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.007e-05 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038185+0000", 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6215000000000001e-05 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038199+0000", 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4018e-05 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.038860+0000", 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00066094500000000002 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.11 125/124 [PushOp(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head, version: 48'115, data_included: [0~262144,524288~287322,819200~49152,942080~49152], data_size: 647770, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head@48'115, size: 1356976, copy_subset: [(0, 1356976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1356976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.3958908870000002, 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002060559, 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135765+0000", 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00092785999999999997 2024-09-17T22:59:16.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135788+0000", 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2623e-05 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135808+0000", 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-05 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:16.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135897+0000", 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8865999999999999e-05 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135909+0000", 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2482e-05 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.136897+0000", 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098805099999999999 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.622 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.623 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:16.623 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:16.623 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:16.624 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:16.625 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:16.622+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:16.725 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:16.723+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:16.767 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:16.767 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:16.768 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:16.768 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:16.776 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:16.826 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:16.824+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:16.878 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:16.878 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:16.878 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:16.878 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:16.878 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:16.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:16.928 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:16.925+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:16.994 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T22:59:17.025 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T22:59:17.028 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:17.025+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 100.781064218, 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.050 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.051 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:59:17.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 100.78072520400001, 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:17.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.058 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 92.603086208999997, 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.059 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:17.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:17.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.065 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958895962, 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:17.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:17.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958873333, 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.071 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:17.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:17.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958673247, 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.076 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:17.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:17.080 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958647675000002, 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:17.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:17.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958466624000003, 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:17.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:17.089 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958431195000003, 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.090 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:17.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.093 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.094 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958332294000002, 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:17.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:17.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:17.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:17.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958156133999999, 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.099 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.958096638000001, 2024-09-17T22:59:17.103 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:17.104 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.105 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:17.107 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947823468999999, 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.108 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.109 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.112 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947735293000001, 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:17.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.114 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.115 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.116 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947682313999998, 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.117 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.118 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.119 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.120 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:17.121 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947624349999998, 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.122 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.123 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.124 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.125 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947340064000002, 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.126 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.127 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:17.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:17.129 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.947163881000002, 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:17.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.132 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.133 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.134 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:17.135 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.946484744000003, 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.136 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.137 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.139 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 60.94646127, 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.140 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:17.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:17.143 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.144 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:17.146 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:17.146 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:17.143+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:17.246 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:17.244+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:17.306 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:17.306 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:17.317 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:17.347 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:17.345+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:17.430 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:17.430 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:17.430 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:17.439 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:17.485 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:17.482+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:17.549 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:17.547+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:17.559 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:17.559 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517245062000001, 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.560 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:17.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.562 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.563 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:17.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.565 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.566 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:17.567 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.569 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517224646999999, 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.570 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.571 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.572 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.573 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517203588000001, 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.574 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:17.576 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.577 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.578 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517180482999997, 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:17.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.580 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.581 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.582 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.583 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517157427000001, 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.584 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.585 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.586 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517126793999999, 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:17.588 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.589 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.590 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517102817999998, 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.593 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:17.594 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:17.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.596 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:17.597 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517072640000002, 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:17.598 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:17.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.600 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.601 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.602 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.517053662999999, 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.603 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.604 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.605 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:17.606 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.516614752000002, 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.607 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.610 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.611 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.516326147000001, 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:17.612 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.613 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:17.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.615 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:17.616 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.515699243, 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.617 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.618 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.619 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.620 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.621 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.515664803999996, 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.622 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:17.624 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.625 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.515276643999997, 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:17.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:17.629 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.513496512000003, 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:17.630 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.631 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.633 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.634 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.513482244000002, 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:17.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.636 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:17.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:17.638 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.513471787999997, 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:17.639 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.640 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:17.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.642 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:17.643 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.513453507000001, 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.644 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.646 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.648 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.513034038999997, 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.649 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:17.650 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:17.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.653 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 61.512407793999998, 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:17.654 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:17.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:17.657 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:17.658 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:17.659 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:17.661 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:17.661 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:17.658+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:17.761 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:17.759+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:17.782 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:17.819 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:17.819 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:17.819 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:17.819 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:17.830 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:17.863 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:17.861+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:17.915 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:17.916 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:17.933 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:17.933 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:17.949 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:17.949 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:17.964 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:17.962+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:17.965 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:17.965 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:17.978 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:17.978 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:17.981 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:17.981 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:17.987 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:17.998 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:17.998 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:18.009 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T22:59:18.018 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:18.018 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:18.033 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T22:59:18.043 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:18.043 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:18.064 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:18.062+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:18.107 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.107 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.116 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:18.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:59:18.165 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:18.163+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:18.221 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.221 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.230 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:18.274 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:18.272+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:18.333 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.334 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.342 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:18.367 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:18.364+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:18.468 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:18.465+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:18.486 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.487 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.495 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:18.569 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:18.566+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:18.599 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.600 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.608 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:18.670 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:18.668+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:18.710 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.710 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:18.770 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:18.768+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:18.822 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.822 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.831 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:18.871 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:18.869+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:18.935 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:18.935 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:18.944 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:18.973 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:18.970+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:19.027 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T22:59:19.038 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:59:19.054 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.054 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.058 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T22:59:19.063 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:19.102 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:19.100+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:19.166 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.167 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.175 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:19.175 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:19.173+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:19.277 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:19.274+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:19.318 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.319 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.328 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:19.377 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:19.375+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:19.432 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.432 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.440 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:19.479 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:19.476+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:19.580 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:19.578+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:19.583 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.583 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:19.681 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:19.679+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:19.701 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.701 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.710 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:19.782 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:19.779+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:19.814 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.815 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.824 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:19.860 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T22:59:19.878 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T22:59:19.883 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:19.881+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:19.975 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:19.975 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:19.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:19.985 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:19.982+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:20.057 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:20.102 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:20.100+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:20.107 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:20.107 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:20.117 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:20.182 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:20.185 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:20.183+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:20.201 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:20.201 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:20.218 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:20.235 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:20.235 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:20.252 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:20.266 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:20.266 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:20.266 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:20.266 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:20.267 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:20.269 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:20.269 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:20.275 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:20.286 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:20.287 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:20.287 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:20.284+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:20.304 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:20.305 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:20.388 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:20.385+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:20.393 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:20.393 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:20.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:20.393 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:20.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16085 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 104.124804256, 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053343962000000002, 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16085 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.394 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265508+0000", 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.27e-07 2024-09-17T22:59:20.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265515+0000", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5020000000000003e-06 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265525+0000", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3589999999999997e-06 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265530+0000", 2024-09-17T22:59:20.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318843+0000", 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053312960999999999 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.318851+0000", 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8489999999999995e-06 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 104.124465242, 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:20.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:20.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 95.946826247000004, 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:20.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:20.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302636000000007, 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:20.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302613371000007, 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:20.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:20.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302413285, 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302387713000002, 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302206662000003, 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302171232999996, 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:20.431 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:20.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:20.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.302072331999995, 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.436 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:20.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.439 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.440 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.301896171999999, 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.441 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.301836675999994, 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.449 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.291563507000006, 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:20.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.291475331000001, 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:20.454 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:20.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.458 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.291422351999998, 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.291364388000005, 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.463 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:20.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:20.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.291080101999995, 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:20.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:20.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:20.472 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.290903919000002, 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:20.476 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.290224781999996, 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.477 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.481 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.290201307999993, 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.485 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.486 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.486 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:20.487 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:20.488 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:20.486+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:20.589 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:20.588+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:20.636 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:20.636 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:20.636 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:20.636 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:20.644 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:20.690 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:20.687+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:20.748 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:20.748 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:20.748 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:20.749 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:20.749 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:20.757 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:20.798 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:20.795+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:20.861 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T22:59:20.880 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T22:59:20.892 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:20.889+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870685171999995, 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:20.912 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.913 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:20.914 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.915 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:20.916 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:20.917 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870664757, 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.918 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:20.919 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.920 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:20.921 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.922 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870643697999995, 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.923 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:20.924 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.925 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:20.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.927 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870620592999998, 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.928 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.929 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:20.930 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:20.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870597536999995, 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.933 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.934 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:20.935 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.936 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.937 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870566904, 2024-09-17T22:59:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.939 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:20.940 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.941 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.942 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:20.943 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.944 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.944 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.945 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.945 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.945 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:20.945 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870542928000006, 2024-09-17T22:59:20.945 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.946 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.947 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:20.947 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.947 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.947 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.948 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:20.950 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.951 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870512750000003, 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.952 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.953 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.954 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:20.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:20.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:20.957 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870493773000007, 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.958 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:20.959 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.960 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.962 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.870054862000003, 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:20.963 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:20.965 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.966 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.967 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.869766256999995, 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:20.968 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.969 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:20.970 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:20.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.972 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.973 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.869139352999994, 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.974 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.975 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.976 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.977 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:20.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.979 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.869104914000005, 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.980 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.982 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.983 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.984 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.985 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.868716754000005, 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.986 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:20.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:20.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.990 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.866936621999997, 2024-09-17T22:59:20.991 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.992 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.993 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:20.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.995 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:20.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.866922353999996, 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:20.997 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:20.998 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:20.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.000 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.001 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.866911897999998, 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:21.002 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.005 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.006 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.007 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.866893617000002, 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:21.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.010 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:21.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.012 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.866474148999998, 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.013 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:21.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:21.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.865847904000006, 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.018 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:21.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:21.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.023 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:21.025 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:21.025 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:21.022+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:21.061 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:59:21.125 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:21.123+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:21.174 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:21.175 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:21.175 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:21.175 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:21.184 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:21.226 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:21.224+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:21.327 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:21.326+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:21.329 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:21.329 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:21.329 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:21.340 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:21.429 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:21.426+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:59:21.446 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.289572675000002, 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:59:21.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:59:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.453 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937784276000002, 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:21.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:59:21.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937763566000001, 2024-09-17T22:59:21.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:59:21.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:59:21.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937743771000001, 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.463 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:21.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:59:21.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937724528000004, 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:59:21.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:59:21.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.472 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937703388000003, 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:59:21.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:59:21.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:59:21.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937678298999998, 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:59:21.477 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:59:21.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:59:21.481 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937658956000007, 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.482 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.483 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:59:21.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:59:21.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:21.486 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937636932999993, 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:21.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:59:21.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937542792000002, 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:59:21.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:59:21.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937516020000004, 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.495 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.497 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.498 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937495494000004, 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.500 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.501 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.502 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937474201000001, 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:59:21.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.506 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.507 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937445326000002, 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.508 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.509 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:59:21.510 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.511 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937404247000003, 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.512 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.513 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.514 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:59:21.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.937380223999995, 2024-09-17T22:59:21.516 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.518 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.519 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.520 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.052637597, 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:21.522 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.523 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:59:21.524 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.052615969000001, 2024-09-17T22:59:21.525 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:59:21.527 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.528 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.529 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.052594784999997, 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.530 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.532 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.534 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 64.052573705, 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.536 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:59:21.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:21.539 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:21.540 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:21.541 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:21.538+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:21.640 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:21.638+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:21.646 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:21.647 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:21.647 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:21.647 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:21.655 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:21.742 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:21.740+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:21.758 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:21.758 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:21.758 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:21.767 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:21.843 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:21.840+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:21.862 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T22:59:21.868 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.486100305999997, 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.875 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:21.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:21.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:21.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.447127354000003, 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:21.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:21.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.447106578000003, 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.885 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.447087788000005, 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:21.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.447042577000005, 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.894 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:21.898 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 64.447008749000005, 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:21.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:21.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.742091481999999, 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:21.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.907 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.742071013999997, 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:21.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.742049453, 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.912 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:21.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.742029635999998, 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.917 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.742009648, 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.921 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:21.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741344235, 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.926 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:21.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.930 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741321489999997, 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:21.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:21.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:21.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:21.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741300357, 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741280240000002, 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:21.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:21.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741253030000003, 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:21.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:21.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:21.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741082169000002, 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:21.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:21.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.741057851000001, 2024-09-17T22:59:21.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:21.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.735783849000001, 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:21.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:21.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200002 3.1b 3:dea2114f:::rbd_object_map.11884120dce1:head [call lock.unlock in=10b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.8772094030000002, 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0273163620000001, 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200002 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.966 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994166+0000", 2024-09-17T22:59:21.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994172+0000", 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2330000000000003e-06 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994181+0000", 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3190000000000001e-06 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:21.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994188+0000", 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8190000000000003e-06 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994239+0000", 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0865000000000002e-05 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994250+0000", 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1191e-05 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019705+0000", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0254550520000001 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019922+0000", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021627500000000001 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020180+0000", 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025870299999999999 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020983+0000", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080232199999999995 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021341+0000", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000358356 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021391+0000", 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0390999999999999e-05 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:21.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021481+0000", 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9673999999999996e-05 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:21.972 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:21.974 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T22:59:21.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:59:21.975 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:21.975 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:21.971+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:22.074 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:22.071+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:22.119 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:22.119 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:22.119 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:22.119 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:22.128 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:22.175 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:22.173+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:22.232 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:22.232 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:22.232 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:22.241 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:22.275 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:22.273+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:22.297 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:22.297 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:59:22.309 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-17T22:59:22.214724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":944805710,"num_objects":1200,"num_object_clones":232,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":625,"num_read":254804,"num_read_kb":790190,"num_write":22351,"num_write_kb":368403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3062,"num_bytes_recovered":105018545,"num_keys_recovered":514,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19318,"ondisk_log_size":19318,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":531884,"kb_used_data":149420,"kb_used_omap":41,"kb_used_meta":380182,"kb_avail":754442836,"statfs":{"total":773094113280,"available":772549464064,"internally_reserved":0,"allocated":153006080,"data_stored":161107479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":389306649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23844727,"num_objects":-17,"num_object_clones":-2,"num_object_copies":-34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-17,"num_whiteouts":-1,"num_read":391,"num_read_kb":345,"num_write":63,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":6862103,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"3.1f","version":"122'439","reported_seq":4474,"reported_epoch":124,"state":"peering","last_fresh":"2024-09-17T22:59:10.950115+0000","last_change":"2024-09-17T22:59:10.950115+0000","last_active":"2024-09-17T22:59:09.948058+0000","last_peered":"2024-09-17T22:59:09.017911+0000","last_clean":"2024-09-17T22:59:09.017911+0000","last_became_active":"2024-09-17T22:58:59.883908+0000","last_became_peered":"2024-09-17T22:58:59.883908+0000","last_unstale":"2024-09-17T22:59:10.950115+0000","last_undegraded":"2024-09-17T22:59:10.950115+0000","last_fullsized":"2024-09-17T22:59:10.950115+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097711999999999994,"stat_sum":{"num_bytes":88042966,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":4242,"num_read_kb":23843,"num_write":441,"num_write_kb":24358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3805168,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"15"}]},{"pgid":"3.1e","version":"130'298","reported_seq":3236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995109+0000","last_change":"2024-09-17T22:59:14.991554+0000","last_active":"2024-09-17T22:59:16.995109+0000","last_peered":"2024-09-17T22:59:16.995109+0000","last_clean":"2024-09-17T22:59:16.995109+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:59:16.995109+0000","last_undegraded":"2024-09-17T22:59:16.995109+0000","last_fullsized":"2024-09-17T22:59:16.995109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:49.656509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412618,"stat_sum":{"num_bytes":25582696,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2647,"num_read_kb":17542,"num_write":273,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.1d","version":"120'534","reported_seq":9694,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867018+0000","last_change":"2024-09-17T22:59:18.017802+0000","last_active":"2024-09-17T22:59:20.867018+0000","last_peered":"2024-09-17T22:59:20.867018+0000","last_clean":"2024-09-17T22:59:20.867018+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T22:59:20.867018+0000","last_undegraded":"2024-09-17T22:59:20.867018+0000","last_fullsized":"2024-09-17T22:59:20.867018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:17:11.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002212122,"stat_sum":{"num_bytes":21602335,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":10247,"num_read_kb":31376,"num_write":558,"num_write_kb":6655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1c","version":"131'539","reported_seq":5651,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866792+0000","last_change":"2024-09-17T22:59:18.018154+0000","last_active":"2024-09-17T22:59:20.866792+0000","last_peered":"2024-09-17T22:59:20.866792+0000","last_clean":"2024-09-17T22:59:20.866792+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T22:59:20.866792+0000","last_undegraded":"2024-09-17T22:59:20.866792+0000","last_fullsized":"2024-09-17T22:59:20.866792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:48.792497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025587560000000001,"stat_sum":{"num_bytes":18517824,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6303,"num_read_kb":16894,"num_write":562,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1b","version":"123'678","reported_seq":14784,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-17T22:59:12.972941+0000","last_change":"2024-09-17T22:59:12.973180+0000","last_active":"2024-09-17T22:59:12.972941+0000","last_peered":"2024-09-17T22:59:12.972941+0000","last_clean":"2024-09-17T22:59:12.972941+0000","last_became_active":"2024-09-17T22:59:01.896650+0000","last_became_peered":"2024-09-17T22:59:01.896650+0000","last_unstale":"2024-09-17T22:59:12.972941+0000","last_undegraded":"2024-09-17T22:59:12.972941+0000","last_fullsized":"2024-09-17T22:59:12.972941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:48:56.955356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072790099999999998,"stat_sum":{"num_bytes":29977018,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":16437,"num_read_kb":56500,"num_write":700,"num_write_kb":16984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":11756698,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"17"}]},{"pgid":"3.1a","version":"130'404","reported_seq":10158,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.054935+0000","last_change":"2024-09-17T22:59:18.010632+0000","last_active":"2024-09-17T22:59:18.054935+0000","last_peered":"2024-09-17T22:59:18.054935+0000","last_clean":"2024-09-17T22:59:18.054935+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:59:18.054935+0000","last_undegraded":"2024-09-17T22:59:18.054935+0000","last_fullsized":"2024-09-17T22:59:18.054935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:42.864990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072336099999999995,"stat_sum":{"num_bytes":21547601,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":10456,"num_read_kb":23168,"num_write":395,"num_write_kb":11995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.19","version":"127'739","reported_seq":5424,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995178+0000","last_change":"2024-09-17T22:59:14.994705+0000","last_active":"2024-09-17T22:59:16.995178+0000","last_peered":"2024-09-17T22:59:16.995178+0000","last_clean":"2024-09-17T22:59:16.995178+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:59:16.995178+0000","last_undegraded":"2024-09-17T22:59:16.995178+0000","last_fullsized":"2024-09-17T22:59:16.995178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:50.293542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054995499999999998,"stat_sum":{"num_bytes":44877072,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":6796,"num_read_kb":19698,"num_write":848,"num_write_kb":4687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.18","version":"123'472","reported_seq":5411,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T22:59:17.021170+0000","last_change":"2024-09-17T22:59:17.021425+0000","last_active":"2024-09-17T22:59:17.021170+0000","last_peered":"2024-09-17T22:59:17.021170+0000","last_clean":"2024-09-17T22:59:13.989336+0000","last_became_active":"2024-09-17T22:59:17.019976+0000","last_became_peered":"2024-09-17T22:59:17.019976+0000","last_unstale":"2024-09-17T22:59:17.021170+0000","last_undegraded":"2024-09-17T22:59:16.996610+0000","last_fullsized":"2024-09-17T22:59:16.995232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046759500000000001,"stat_sum":{"num_bytes":38067610,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":4737,"num_read_kb":27532,"num_write":456,"num_write_kb":27831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5577430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":43}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.17","version":"130'662","reported_seq":4513,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.132445+0000","last_change":"2024-09-17T22:59:14.990451+0000","last_active":"2024-09-17T22:59:17.132445+0000","last_peered":"2024-09-17T22:59:17.132445+0000","last_clean":"2024-09-17T22:59:17.132445+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T22:59:17.132445+0000","last_undegraded":"2024-09-17T22:59:17.132445+0000","last_fullsized":"2024-09-17T22:59:17.132445+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:06.424566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067086800000000003,"stat_sum":{"num_bytes":37126897,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":5568,"num_read_kb":21111,"num_write":799,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.16","version":"127'1303","reported_seq":9540,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866783+0000","last_change":"2024-09-17T22:59:18.017250+0000","last_active":"2024-09-17T22:59:20.866783+0000","last_peered":"2024-09-17T22:59:20.866783+0000","last_clean":"2024-09-17T22:59:20.866783+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T22:59:20.866783+0000","last_undegraded":"2024-09-17T22:59:20.866783+0000","last_fullsized":"2024-09-17T22:59:20.866783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1303,"log_dups_size":0,"ondisk_log_size":1303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:39.632576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695525,"stat_sum":{"num_bytes":40767942,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":13640,"num_read_kb":44259,"num_write":1892,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.15","version":"127'433","reported_seq":4686,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.996621+0000","last_change":"2024-09-17T22:59:14.991876+0000","last_active":"2024-09-17T22:59:16.996621+0000","last_peered":"2024-09-17T22:59:16.996621+0000","last_clean":"2024-09-17T22:59:16.996621+0000","last_became_active":"2024-09-17T22:58:46.132716+0000","last_became_peered":"2024-09-17T22:58:46.132716+0000","last_unstale":"2024-09-17T22:59:16.996621+0000","last_undegraded":"2024-09-17T22:59:16.996621+0000","last_fullsized":"2024-09-17T22:59:16.996621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:02.761379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012511930000000001,"stat_sum":{"num_bytes":63637947,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":4517,"num_read_kb":28303,"num_write":404,"num_write_kb":9577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.14","version":"130'462","reported_seq":2769,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.010695+0000","last_change":"2024-09-17T22:59:18.011299+0000","last_active":"2024-09-17T22:59:18.010695+0000","last_peered":"2024-09-17T22:59:18.010695+0000","last_clean":"2024-09-17T22:59:18.010695+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:59:18.010695+0000","last_undegraded":"2024-09-17T22:59:18.010695+0000","last_fullsized":"2024-09-17T22:59:18.010695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:12.990255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689870000000001,"stat_sum":{"num_bytes":15554461,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":3496,"num_read_kb":11431,"num_write":451,"num_write_kb":5675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.014877+0000","last_change":"2024-09-17T22:59:11.956325+0000","last_active":"2024-09-17T22:59:18.014877+0000","last_peered":"2024-09-17T22:59:18.014877+0000","last_clean":"2024-09-17T22:59:18.014877+0000","last_became_active":"2024-09-17T22:59:11.956025+0000","last_became_peered":"2024-09-17T22:59:11.956025+0000","last_unstale":"2024-09-17T22:59:18.014877+0000","last_undegraded":"2024-09-17T22:59:18.014877+0000","last_fullsized":"2024-09-17T22:59:18.014877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:21:59.355000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"130'601","reported_seq":9990,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.087711+0000","last_change":"2024-09-17T22:59:14.990812+0000","last_active":"2024-09-17T22:59:17.087711+0000","last_peered":"2024-09-17T22:59:17.087711+0000","last_clean":"2024-09-17T22:59:17.087711+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:59:17.087711+0000","last_undegraded":"2024-09-17T22:59:17.087711+0000","last_fullsized":"2024-09-17T22:59:17.087711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:37.624706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041631099999999999,"stat_sum":{"num_bytes":47054077,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":11985,"num_read_kb":26141,"num_write":527,"num_write_kb":10122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.6","version":"0'0","reported_seq":237,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994878+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:59:16.994878+0000","last_peered":"2024-09-17T22:59:16.994878+0000","last_clean":"2024-09-17T22:59:16.994878+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:59:16.994878+0000","last_undegraded":"2024-09-17T22:59:16.994878+0000","last_fullsized":"2024-09-17T22:59:16.994878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"127'483","reported_seq":9566,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994935+0000","last_change":"2024-09-17T22:59:14.991883+0000","last_active":"2024-09-17T22:59:16.994935+0000","last_peered":"2024-09-17T22:59:16.994935+0000","last_clean":"2024-09-17T22:59:16.994935+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:59:16.994935+0000","last_undegraded":"2024-09-17T22:59:16.994935+0000","last_fullsized":"2024-09-17T22:59:16.994935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:01.318468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015285780000000001,"stat_sum":{"num_bytes":12802556,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":9662,"num_read_kb":29730,"num_write":502,"num_write_kb":5475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866222+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:59:20.866222+0000","last_peered":"2024-09-17T22:59:20.866222+0000","last_clean":"2024-09-17T22:59:20.866222+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:59:20.866222+0000","last_undegraded":"2024-09-17T22:59:20.866222+0000","last_fullsized":"2024-09-17T22:59:20.866222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"130'509","reported_seq":9593,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995023+0000","last_change":"2024-09-17T22:59:14.995548+0000","last_active":"2024-09-17T22:59:16.995023+0000","last_peered":"2024-09-17T22:59:16.995023+0000","last_clean":"2024-09-17T22:59:16.995023+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T22:59:16.995023+0000","last_undegraded":"2024-09-17T22:59:16.995023+0000","last_fullsized":"2024-09-17T22:59:16.995023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:37.873151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072807199999999996,"stat_sum":{"num_bytes":10497666,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":10330,"num_read_kb":26999,"num_write":488,"num_write_kb":10537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995112+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:59:16.995112+0000","last_peered":"2024-09-17T22:59:16.995112+0000","last_clean":"2024-09-17T22:59:16.995112+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:59:16.995112+0000","last_undegraded":"2024-09-17T22:59:16.995112+0000","last_fullsized":"2024-09-17T22:59:16.995112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"130'703","reported_seq":3755,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866901+0000","last_change":"2024-09-17T22:59:18.016216+0000","last_active":"2024-09-17T22:59:20.866901+0000","last_peered":"2024-09-17T22:59:20.866901+0000","last_clean":"2024-09-17T22:59:20.866901+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T22:59:20.866901+0000","last_undegraded":"2024-09-17T22:59:20.866901+0000","last_fullsized":"2024-09-17T22:59:20.866901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:38.503327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091405800000000004,"stat_sum":{"num_bytes":16532738,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":5442,"num_read_kb":19069,"num_write":877,"num_write_kb":12167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.3","version":"131'605","reported_seq":4964,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.073061+0000","last_change":"2024-09-17T22:59:18.012038+0000","last_active":"2024-09-17T22:59:18.073061+0000","last_peered":"2024-09-17T22:59:18.073061+0000","last_clean":"2024-09-17T22:59:18.073061+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:59:18.073061+0000","last_undegraded":"2024-09-17T22:59:18.073061+0000","last_fullsized":"2024-09-17T22:59:18.073061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:12.453871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001499923,"stat_sum":{"num_bytes":42378885,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":5114,"num_read_kb":23733,"num_write":607,"num_write_kb":9693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866652+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T22:59:20.866652+0000","last_peered":"2024-09-17T22:59:20.866652+0000","last_clean":"2024-09-17T22:59:20.866652+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T22:59:20.866652+0000","last_undegraded":"2024-09-17T22:59:20.866652+0000","last_fullsized":"2024-09-17T22:59:20.866652+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"130'365","reported_seq":18950,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.135736+0000","last_change":"2024-09-17T22:59:14.996187+0000","last_active":"2024-09-17T22:59:17.135736+0000","last_peered":"2024-09-17T22:59:17.135736+0000","last_clean":"2024-09-17T22:59:17.135736+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T22:59:17.135736+0000","last_undegraded":"2024-09-17T22:59:17.135736+0000","last_fullsized":"2024-09-17T22:59:17.135736+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:37.581054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045200999999999998,"stat_sum":{"num_bytes":28389176,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":19387,"num_read_kb":25736,"num_write":332,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.015914+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T22:59:18.015914+0000","last_peered":"2024-09-17T22:59:18.015914+0000","last_clean":"2024-09-17T22:59:18.015914+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T22:59:18.015914+0000","last_undegraded":"2024-09-17T22:59:18.015914+0000","last_fullsized":"2024-09-17T22:59:18.015914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"130'1215","reported_seq":12823,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.997984+0000","last_change":"2024-09-17T22:59:14.991025+0000","last_active":"2024-09-17T22:59:16.997984+0000","last_peered":"2024-09-17T22:59:16.997984+0000","last_clean":"2024-09-17T22:59:16.997984+0000","last_became_active":"2024-09-17T22:59:01.898399+0000","last_became_peered":"2024-09-17T22:59:01.898399+0000","last_unstale":"2024-09-17T22:59:16.997984+0000","last_undegraded":"2024-09-17T22:59:16.997984+0000","last_fullsized":"2024-09-17T22:59:16.997984+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1215,"log_dups_size":0,"ondisk_log_size":1215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:06.772814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044412599999999998,"stat_sum":{"num_bytes":17399392,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":16140,"num_read_kb":43091,"num_write":1871,"num_write_kb":4570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":674738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995493+0000","last_change":"2024-09-17T22:58:46.131382+0000","last_active":"2024-09-17T22:59:16.995493+0000","last_peered":"2024-09-17T22:59:16.995493+0000","last_clean":"2024-09-17T22:59:16.995493+0000","last_became_active":"2024-09-17T22:58:46.130955+0000","last_became_peered":"2024-09-17T22:58:46.130955+0000","last_unstale":"2024-09-17T22:59:16.995493+0000","last_undegraded":"2024-09-17T22:59:16.995493+0000","last_fullsized":"2024-09-17T22:59:16.995493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.813197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"121'815","reported_seq":6030,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994741+0000","last_change":"2024-09-17T22:59:14.991517+0000","last_active":"2024-09-17T22:59:16.994741+0000","last_peered":"2024-09-17T22:59:16.994741+0000","last_clean":"2024-09-17T22:59:16.994741+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T22:59:16.994741+0000","last_undegraded":"2024-09-17T22:59:16.994741+0000","last_fullsized":"2024-09-17T22:59:16.994741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:56:23.068486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001702505,"stat_sum":{"num_bytes":18865494,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":6951,"num_read_kb":15969,"num_write":1051,"num_write_kb":7549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.3","version":"19'1","reported_seq":75,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866696+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T22:59:20.866696+0000","last_peered":"2024-09-17T22:59:20.866696+0000","last_clean":"2024-09-17T22:59:20.866696+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T22:59:20.866696+0000","last_undegraded":"2024-09-17T22:59:20.866696+0000","last_fullsized":"2024-09-17T22:59:20.866696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.998202+0000","last_change":"2024-09-17T22:58:46.130584+0000","last_active":"2024-09-17T22:59:16.998202+0000","last_peered":"2024-09-17T22:59:16.998202+0000","last_clean":"2024-09-17T22:59:16.998202+0000","last_became_active":"2024-09-17T22:58:46.130202+0000","last_became_peered":"2024-09-17T22:58:46.130202+0000","last_unstale":"2024-09-17T22:59:16.998202+0000","last_undegraded":"2024-09-17T22:59:16.998202+0000","last_fullsized":"2024-09-17T22:59:16.998202+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:32.916711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"130'788","reported_seq":3735,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995347+0000","last_change":"2024-09-17T22:59:14.995614+0000","last_active":"2024-09-17T22:59:16.995347+0000","last_peered":"2024-09-17T22:59:16.995347+0000","last_clean":"2024-09-17T22:59:16.995347+0000","last_became_active":"2024-09-17T22:59:01.896831+0000","last_became_peered":"2024-09-17T22:59:01.896831+0000","last_unstale":"2024-09-17T22:59:16.995347+0000","last_undegraded":"2024-09-17T22:59:16.995347+0000","last_fullsized":"2024-09-17T22:59:16.995347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:29.635477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079523300000000001,"stat_sum":{"num_bytes":42088635,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":5344,"num_read_kb":13669,"num_write":1097,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3744943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.9","version":"130'505","reported_seq":3957,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.016920+0000","last_change":"2024-09-17T22:59:18.017416+0000","last_active":"2024-09-17T22:59:18.016920+0000","last_peered":"2024-09-17T22:59:18.016920+0000","last_clean":"2024-09-17T22:59:18.016920+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T22:59:18.016920+0000","last_undegraded":"2024-09-17T22:59:18.016920+0000","last_fullsized":"2024-09-17T22:59:18.016920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:55:46.203492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067207599999999997,"stat_sum":{"num_bytes":10471373,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5160,"num_read_kb":14284,"num_write":492,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.a","version":"131'526","reported_seq":3421,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.088546+0000","last_change":"2024-09-17T22:59:18.020579+0000","last_active":"2024-09-17T22:59:18.088546+0000","last_peered":"2024-09-17T22:59:18.088546+0000","last_clean":"2024-09-17T22:59:18.088546+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:59:18.088546+0000","last_undegraded":"2024-09-17T22:59:18.088546+0000","last_fullsized":"2024-09-17T22:59:18.088546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:08.003724+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003826075,"stat_sum":{"num_bytes":8015804,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":3596,"num_read_kb":19037,"num_write":619,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.b","version":"127'510","reported_seq":6142,"reported_epoch":130,"state":"active+undersized+remapped","last_fresh":"2024-09-17T22:59:17.186375+0000","last_change":"2024-09-17T22:59:17.186583+0000","last_active":"2024-09-17T22:59:17.186375+0000","last_peered":"2024-09-17T22:59:17.186375+0000","last_clean":"2024-09-17T22:59:14.059022+0000","last_became_active":"2024-09-17T22:59:17.018829+0000","last_became_peered":"2024-09-17T22:59:17.018829+0000","last_unstale":"2024-09-17T22:59:17.186375+0000","last_undegraded":"2024-09-17T22:59:17.186375+0000","last_fullsized":"2024-09-17T22:59:16.994496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090487300000000005,"stat_sum":{"num_bytes":15510395,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":6277,"num_read_kb":16384,"num_write":513,"num_write_kb":10978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","0","4"],"object_location_counts":[{"shards":"3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.c","version":"127'403","reported_seq":5020,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866755+0000","last_change":"2024-09-17T22:59:18.016581+0000","last_active":"2024-09-17T22:59:20.866755+0000","last_peered":"2024-09-17T22:59:20.866755+0000","last_clean":"2024-09-17T22:59:20.866755+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T22:59:20.866755+0000","last_undegraded":"2024-09-17T22:59:20.866755+0000","last_fullsized":"2024-09-17T22:59:20.866755+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:34.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652049999999999,"stat_sum":{"num_bytes":16549431,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":5438,"num_read_kb":18886,"num_write":470,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.d","version":"130'369","reported_seq":4573,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866910+0000","last_change":"2024-09-17T22:59:18.017682+0000","last_active":"2024-09-17T22:59:20.866910+0000","last_peered":"2024-09-17T22:59:20.866910+0000","last_clean":"2024-09-17T22:59:20.866910+0000","last_became_active":"2024-09-17T22:58:54.861414+0000","last_became_peered":"2024-09-17T22:58:54.861414+0000","last_unstale":"2024-09-17T22:59:20.866910+0000","last_undegraded":"2024-09-17T22:59:20.866910+0000","last_fullsized":"2024-09-17T22:59:20.866910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:14.726671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021113640000000001,"stat_sum":{"num_bytes":21422337,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5562,"num_read_kb":23049,"num_write":382,"num_write_kb":6583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.e","version":"127'323","reported_seq":1963,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994713+0000","last_change":"2024-09-17T22:59:14.995183+0000","last_active":"2024-09-17T22:59:16.994713+0000","last_peered":"2024-09-17T22:59:16.994713+0000","last_clean":"2024-09-17T22:59:16.994713+0000","last_became_active":"2024-09-17T22:59:01.896809+0000","last_became_peered":"2024-09-17T22:59:01.896809+0000","last_unstale":"2024-09-17T22:59:16.994713+0000","last_undegraded":"2024-09-17T22:59:16.994713+0000","last_fullsized":"2024-09-17T22:59:16.994713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:19:43.185212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060952000000000001,"stat_sum":{"num_bytes":12542734,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":1556,"num_read_kb":7770,"num_write":312,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2492972,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.f","version":"131'1021","reported_seq":8992,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.012564+0000","last_change":"2024-09-17T22:59:18.012564+0000","last_active":"2024-09-17T22:59:18.012564+0000","last_peered":"2024-09-17T22:59:18.012564+0000","last_clean":"2024-09-17T22:59:18.012564+0000","last_became_active":"2024-09-17T22:58:46.132674+0000","last_became_peered":"2024-09-17T22:58:46.132674+0000","last_unstale":"2024-09-17T22:59:18.012564+0000","last_undegraded":"2024-09-17T22:59:18.012564+0000","last_fullsized":"2024-09-17T22:59:18.012564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:52.449599+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0025620830000000002,"stat_sum":{"num_bytes":38757988,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":12100,"num_read_kb":34693,"num_write":1339,"num_write_kb":49269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.10","version":"127'644","reported_seq":5530,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-17T22:59:17.719756+0000","last_change":"2024-09-17T22:59:18.015342+0000","last_active":"2024-09-17T22:59:18.015342+0000","last_peered":"2024-09-17T22:59:17.719756+0000","last_clean":"2024-09-17T22:59:14.060717+0000","last_became_active":"2024-09-17T22:59:17.019737+0000","last_became_peered":"2024-09-17T22:59:17.019737+0000","last_unstale":"2024-09-17T22:59:17.719756+0000","last_undegraded":"2024-09-17T22:59:17.719756+0000","last_fullsized":"2024-09-17T22:59:16.998730+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048484400000000001,"stat_sum":{"num_bytes":65332590,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":5655,"num_read_kb":33513,"num_write":837,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.11","version":"131'490","reported_seq":4355,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867198+0000","last_change":"2024-09-17T22:59:18.016878+0000","last_active":"2024-09-17T22:59:20.867198+0000","last_peered":"2024-09-17T22:59:20.867198+0000","last_clean":"2024-09-17T22:59:20.867198+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T22:59:20.867198+0000","last_undegraded":"2024-09-17T22:59:20.867198+0000","last_fullsized":"2024-09-17T22:59:20.867198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:07.569383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647997,"stat_sum":{"num_bytes":13042207,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":4434,"num_read_kb":17010,"num_write":467,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.12","version":"120'753","reported_seq":13206,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.998060+0000","last_change":"2024-09-17T22:59:14.990249+0000","last_active":"2024-09-17T22:59:16.998060+0000","last_peered":"2024-09-17T22:59:16.998060+0000","last_clean":"2024-09-17T22:59:16.998060+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:59:16.998060+0000","last_undegraded":"2024-09-17T22:59:16.998060+0000","last_fullsized":"2024-09-17T22:59:16.998060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:22:24.860341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049187600000000003,"stat_sum":{"num_bytes":26054365,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":15096,"num_read_kb":33022,"num_write":952,"num_write_kb":9478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.13","version":"130'607","reported_seq":8883,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866147+0000","last_change":"2024-09-17T22:59:18.012873+0000","last_active":"2024-09-17T22:59:20.866147+0000","last_peered":"2024-09-17T22:59:20.866147+0000","last_clean":"2024-09-17T22:59:20.866147+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:59:20.866147+0000","last_undegraded":"2024-09-17T22:59:20.866147+0000","last_fullsized":"2024-09-17T22:59:20.866147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:19.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074441799999999995,"stat_sum":{"num_bytes":35205111,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":10373,"num_read_kb":26648,"num_write":676,"num_write_kb":10339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":944215323,"num_objects":1196,"num_object_clones":232,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":625,"num_read":254688,"num_read_kb":790090,"num_write":22190,"num_write_kb":365985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":105018507,"num_keys_recovered":514,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135028736,"data_stored":149542007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":19198,"ondisk_log_size":19198,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50668,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321172,"statfs":{"total":96636764160,"available":96584880128,"internally_reserved":0,"allocated":11427840,"data_stored":11394196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678032,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63612,"kb_used_data":22308,"kb_used_omap":17,"kb_used_meta":41262,"kb_avail":94308228,"statfs":{"total":96636764160,"available":96571625472,"internally_reserved":0,"allocated":22843392,"data_stored":23898875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":42252486},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90184,"kb_used_data":30568,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94281656,"statfs":{"total":96636764160,"available":96544415744,"internally_reserved":0,"allocated":31301632,"data_stored":34393366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74692,"kb_used_data":21284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297148,"statfs":{"total":96636764160,"available":96560279552,"internally_reserved":0,"allocated":21794816,"data_stored":23839469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977411,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":9348,"kb_used_omap":14,"kb_used_meta":28401,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":9572352,"data_stored":9610511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":29083050},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68044,"kb_used_data":18808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94303796,"statfs":{"total":96636764160,"available":96567087104,"internally_reserved":0,"allocated":19259392,"data_stored":19397216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62368,"kb_used_data":8968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309472,"statfs":{"total":96636764160,"available":96572899328,"internally_reserved":0,"allocated":9183232,"data_stored":8452450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84532,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287308,"statfs":{"total":96636764160,"available":96550203392,"internally_reserved":0,"allocated":27623424,"data_stored":30121396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7626752,"data_stored":7622593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15077376,"data_stored":16023503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":8856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18141184,"data_stored":20986702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29679616,"data_stored":33581478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21241856,"data_stored":23126463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9818112,"data_stored":10586377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25432064,"data_stored":28758635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:59:22.311 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:59:22.317 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.09031907000001, 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:22.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:22.351 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.090303065, 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:22.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:22.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.090270295, 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:22.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.090242557, 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.361 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:22.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.09022013800001, 2024-09-17T22:59:22.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:22.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.090197858, 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:22.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.089990453, 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:22.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.08944383799999, 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.379 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:22.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:22.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.088372642, 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.384 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:22.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:22.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.388 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.088361187, 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.389 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:22.393 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.08835021, 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:22.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:22.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.086276078, 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.398 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:22.402 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.08626376399999, 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.403 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:22.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.086250527, 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:22.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:22.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.086239698, 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.412 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:22.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:22.416 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.084280823, 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.084256465, 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.421 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:22.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:22.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.084227247, 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.426 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:22.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 106.083447577, 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.432 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:22.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.464589039, 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.437 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:22.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:22.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:22.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:22.444 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:22.446 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:22.447 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:22.443+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:22.547 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:22.544+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:22.552 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:22.552 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:22.552 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:22.561 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:22.568 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:22.569 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:22.586 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:22.586 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:22.602 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:22.602 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:22.619 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:22.619 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:22.637 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:22.648 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:22.645+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:22.653 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:22.669 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:22.669 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:22.685 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:22.686 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:22.707 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:22.708 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:22.717 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:22.735 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:22.735 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:59:22.747 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-17T22:59:22.214724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":944805710,"num_objects":1200,"num_object_clones":232,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":625,"num_read":254804,"num_read_kb":790190,"num_write":22351,"num_write_kb":368403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3062,"num_bytes_recovered":105018545,"num_keys_recovered":514,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19318,"ondisk_log_size":19318,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":531884,"kb_used_data":149420,"kb_used_omap":41,"kb_used_meta":380182,"kb_avail":754442836,"statfs":{"total":773094113280,"available":772549464064,"internally_reserved":0,"allocated":153006080,"data_stored":161107479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":389306649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23844727,"num_objects":-17,"num_object_clones":-2,"num_object_copies":-34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-17,"num_whiteouts":-1,"num_read":391,"num_read_kb":345,"num_write":63,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":6862103,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"3.1f","version":"122'439","reported_seq":4474,"reported_epoch":124,"state":"peering","last_fresh":"2024-09-17T22:59:10.950115+0000","last_change":"2024-09-17T22:59:10.950115+0000","last_active":"2024-09-17T22:59:09.948058+0000","last_peered":"2024-09-17T22:59:09.017911+0000","last_clean":"2024-09-17T22:59:09.017911+0000","last_became_active":"2024-09-17T22:58:59.883908+0000","last_became_peered":"2024-09-17T22:58:59.883908+0000","last_unstale":"2024-09-17T22:59:10.950115+0000","last_undegraded":"2024-09-17T22:59:10.950115+0000","last_fullsized":"2024-09-17T22:59:10.950115+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097711999999999994,"stat_sum":{"num_bytes":88042966,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":4242,"num_read_kb":23843,"num_write":441,"num_write_kb":24358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3805168,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"15"}]},{"pgid":"3.1e","version":"130'298","reported_seq":3236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995109+0000","last_change":"2024-09-17T22:59:14.991554+0000","last_active":"2024-09-17T22:59:16.995109+0000","last_peered":"2024-09-17T22:59:16.995109+0000","last_clean":"2024-09-17T22:59:16.995109+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:59:16.995109+0000","last_undegraded":"2024-09-17T22:59:16.995109+0000","last_fullsized":"2024-09-17T22:59:16.995109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:49.656509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412618,"stat_sum":{"num_bytes":25582696,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2647,"num_read_kb":17542,"num_write":273,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.1d","version":"120'534","reported_seq":9694,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867018+0000","last_change":"2024-09-17T22:59:18.017802+0000","last_active":"2024-09-17T22:59:20.867018+0000","last_peered":"2024-09-17T22:59:20.867018+0000","last_clean":"2024-09-17T22:59:20.867018+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T22:59:20.867018+0000","last_undegraded":"2024-09-17T22:59:20.867018+0000","last_fullsized":"2024-09-17T22:59:20.867018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:17:11.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002212122,"stat_sum":{"num_bytes":21602335,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":10247,"num_read_kb":31376,"num_write":558,"num_write_kb":6655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1c","version":"131'539","reported_seq":5651,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866792+0000","last_change":"2024-09-17T22:59:18.018154+0000","last_active":"2024-09-17T22:59:20.866792+0000","last_peered":"2024-09-17T22:59:20.866792+0000","last_clean":"2024-09-17T22:59:20.866792+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T22:59:20.866792+0000","last_undegraded":"2024-09-17T22:59:20.866792+0000","last_fullsized":"2024-09-17T22:59:20.866792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:48.792497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025587560000000001,"stat_sum":{"num_bytes":18517824,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6303,"num_read_kb":16894,"num_write":562,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1b","version":"123'678","reported_seq":14784,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-17T22:59:12.972941+0000","last_change":"2024-09-17T22:59:12.973180+0000","last_active":"2024-09-17T22:59:12.972941+0000","last_peered":"2024-09-17T22:59:12.972941+0000","last_clean":"2024-09-17T22:59:12.972941+0000","last_became_active":"2024-09-17T22:59:01.896650+0000","last_became_peered":"2024-09-17T22:59:01.896650+0000","last_unstale":"2024-09-17T22:59:12.972941+0000","last_undegraded":"2024-09-17T22:59:12.972941+0000","last_fullsized":"2024-09-17T22:59:12.972941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:48:56.955356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072790099999999998,"stat_sum":{"num_bytes":29977018,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":16437,"num_read_kb":56500,"num_write":700,"num_write_kb":16984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":11756698,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"17"}]},{"pgid":"3.1a","version":"130'404","reported_seq":10158,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.054935+0000","last_change":"2024-09-17T22:59:18.010632+0000","last_active":"2024-09-17T22:59:18.054935+0000","last_peered":"2024-09-17T22:59:18.054935+0000","last_clean":"2024-09-17T22:59:18.054935+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:59:18.054935+0000","last_undegraded":"2024-09-17T22:59:18.054935+0000","last_fullsized":"2024-09-17T22:59:18.054935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:42.864990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072336099999999995,"stat_sum":{"num_bytes":21547601,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":10456,"num_read_kb":23168,"num_write":395,"num_write_kb":11995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.19","version":"127'739","reported_seq":5424,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995178+0000","last_change":"2024-09-17T22:59:14.994705+0000","last_active":"2024-09-17T22:59:16.995178+0000","last_peered":"2024-09-17T22:59:16.995178+0000","last_clean":"2024-09-17T22:59:16.995178+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:59:16.995178+0000","last_undegraded":"2024-09-17T22:59:16.995178+0000","last_fullsized":"2024-09-17T22:59:16.995178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:50.293542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054995499999999998,"stat_sum":{"num_bytes":44877072,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":6796,"num_read_kb":19698,"num_write":848,"num_write_kb":4687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.18","version":"123'472","reported_seq":5411,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T22:59:17.021170+0000","last_change":"2024-09-17T22:59:17.021425+0000","last_active":"2024-09-17T22:59:17.021170+0000","last_peered":"2024-09-17T22:59:17.021170+0000","last_clean":"2024-09-17T22:59:13.989336+0000","last_became_active":"2024-09-17T22:59:17.019976+0000","last_became_peered":"2024-09-17T22:59:17.019976+0000","last_unstale":"2024-09-17T22:59:17.021170+0000","last_undegraded":"2024-09-17T22:59:16.996610+0000","last_fullsized":"2024-09-17T22:59:16.995232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046759500000000001,"stat_sum":{"num_bytes":38067610,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":4737,"num_read_kb":27532,"num_write":456,"num_write_kb":27831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5577430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":43}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.17","version":"130'662","reported_seq":4513,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.132445+0000","last_change":"2024-09-17T22:59:14.990451+0000","last_active":"2024-09-17T22:59:17.132445+0000","last_peered":"2024-09-17T22:59:17.132445+0000","last_clean":"2024-09-17T22:59:17.132445+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T22:59:17.132445+0000","last_undegraded":"2024-09-17T22:59:17.132445+0000","last_fullsized":"2024-09-17T22:59:17.132445+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:06.424566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067086800000000003,"stat_sum":{"num_bytes":37126897,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":5568,"num_read_kb":21111,"num_write":799,"num_write_kb":11454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.16","version":"127'1303","reported_seq":9540,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866783+0000","last_change":"2024-09-17T22:59:18.017250+0000","last_active":"2024-09-17T22:59:20.866783+0000","last_peered":"2024-09-17T22:59:20.866783+0000","last_clean":"2024-09-17T22:59:20.866783+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T22:59:20.866783+0000","last_undegraded":"2024-09-17T22:59:20.866783+0000","last_fullsized":"2024-09-17T22:59:20.866783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1303,"log_dups_size":0,"ondisk_log_size":1303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:39.632576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695525,"stat_sum":{"num_bytes":40767942,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":13640,"num_read_kb":44259,"num_write":1892,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.15","version":"127'433","reported_seq":4686,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.996621+0000","last_change":"2024-09-17T22:59:14.991876+0000","last_active":"2024-09-17T22:59:16.996621+0000","last_peered":"2024-09-17T22:59:16.996621+0000","last_clean":"2024-09-17T22:59:16.996621+0000","last_became_active":"2024-09-17T22:58:46.132716+0000","last_became_peered":"2024-09-17T22:58:46.132716+0000","last_unstale":"2024-09-17T22:59:16.996621+0000","last_undegraded":"2024-09-17T22:59:16.996621+0000","last_fullsized":"2024-09-17T22:59:16.996621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:02.761379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012511930000000001,"stat_sum":{"num_bytes":63637947,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":4517,"num_read_kb":28303,"num_write":404,"num_write_kb":9577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.14","version":"130'462","reported_seq":2769,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.010695+0000","last_change":"2024-09-17T22:59:18.011299+0000","last_active":"2024-09-17T22:59:18.010695+0000","last_peered":"2024-09-17T22:59:18.010695+0000","last_clean":"2024-09-17T22:59:18.010695+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:59:18.010695+0000","last_undegraded":"2024-09-17T22:59:18.010695+0000","last_fullsized":"2024-09-17T22:59:18.010695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:12.990255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689870000000001,"stat_sum":{"num_bytes":15554461,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":3496,"num_read_kb":11431,"num_write":451,"num_write_kb":5675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.014877+0000","last_change":"2024-09-17T22:59:11.956325+0000","last_active":"2024-09-17T22:59:18.014877+0000","last_peered":"2024-09-17T22:59:18.014877+0000","last_clean":"2024-09-17T22:59:18.014877+0000","last_became_active":"2024-09-17T22:59:11.956025+0000","last_became_peered":"2024-09-17T22:59:11.956025+0000","last_unstale":"2024-09-17T22:59:18.014877+0000","last_undegraded":"2024-09-17T22:59:18.014877+0000","last_fullsized":"2024-09-17T22:59:18.014877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:21:59.355000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"130'601","reported_seq":9990,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.087711+0000","last_change":"2024-09-17T22:59:14.990812+0000","last_active":"2024-09-17T22:59:17.087711+0000","last_peered":"2024-09-17T22:59:17.087711+0000","last_clean":"2024-09-17T22:59:17.087711+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:59:17.087711+0000","last_undegraded":"2024-09-17T22:59:17.087711+0000","last_fullsized":"2024-09-17T22:59:17.087711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:30:37.624706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041631099999999999,"stat_sum":{"num_bytes":47054077,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":11985,"num_read_kb":26141,"num_write":527,"num_write_kb":10122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.6","version":"0'0","reported_seq":237,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994878+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:59:16.994878+0000","last_peered":"2024-09-17T22:59:16.994878+0000","last_clean":"2024-09-17T22:59:16.994878+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:59:16.994878+0000","last_undegraded":"2024-09-17T22:59:16.994878+0000","last_fullsized":"2024-09-17T22:59:16.994878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"127'483","reported_seq":9566,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994935+0000","last_change":"2024-09-17T22:59:14.991883+0000","last_active":"2024-09-17T22:59:16.994935+0000","last_peered":"2024-09-17T22:59:16.994935+0000","last_clean":"2024-09-17T22:59:16.994935+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:59:16.994935+0000","last_undegraded":"2024-09-17T22:59:16.994935+0000","last_fullsized":"2024-09-17T22:59:16.994935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:01.318468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015285780000000001,"stat_sum":{"num_bytes":12802556,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":9662,"num_read_kb":29730,"num_write":502,"num_write_kb":5475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866222+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:59:20.866222+0000","last_peered":"2024-09-17T22:59:20.866222+0000","last_clean":"2024-09-17T22:59:20.866222+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:59:20.866222+0000","last_undegraded":"2024-09-17T22:59:20.866222+0000","last_fullsized":"2024-09-17T22:59:20.866222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"130'509","reported_seq":9593,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995023+0000","last_change":"2024-09-17T22:59:14.995548+0000","last_active":"2024-09-17T22:59:16.995023+0000","last_peered":"2024-09-17T22:59:16.995023+0000","last_clean":"2024-09-17T22:59:16.995023+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T22:59:16.995023+0000","last_undegraded":"2024-09-17T22:59:16.995023+0000","last_fullsized":"2024-09-17T22:59:16.995023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:37.873151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072807199999999996,"stat_sum":{"num_bytes":10497666,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":10330,"num_read_kb":26999,"num_write":488,"num_write_kb":10537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995112+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:59:16.995112+0000","last_peered":"2024-09-17T22:59:16.995112+0000","last_clean":"2024-09-17T22:59:16.995112+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:59:16.995112+0000","last_undegraded":"2024-09-17T22:59:16.995112+0000","last_fullsized":"2024-09-17T22:59:16.995112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"130'703","reported_seq":3755,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866901+0000","last_change":"2024-09-17T22:59:18.016216+0000","last_active":"2024-09-17T22:59:20.866901+0000","last_peered":"2024-09-17T22:59:20.866901+0000","last_clean":"2024-09-17T22:59:20.866901+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T22:59:20.866901+0000","last_undegraded":"2024-09-17T22:59:20.866901+0000","last_fullsized":"2024-09-17T22:59:20.866901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:38.503327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091405800000000004,"stat_sum":{"num_bytes":16532738,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":5442,"num_read_kb":19069,"num_write":877,"num_write_kb":12167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.3","version":"131'605","reported_seq":4964,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.073061+0000","last_change":"2024-09-17T22:59:18.012038+0000","last_active":"2024-09-17T22:59:18.073061+0000","last_peered":"2024-09-17T22:59:18.073061+0000","last_clean":"2024-09-17T22:59:18.073061+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:59:18.073061+0000","last_undegraded":"2024-09-17T22:59:18.073061+0000","last_fullsized":"2024-09-17T22:59:18.073061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:12.453871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001499923,"stat_sum":{"num_bytes":42378885,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":5114,"num_read_kb":23733,"num_write":607,"num_write_kb":9693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866652+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T22:59:20.866652+0000","last_peered":"2024-09-17T22:59:20.866652+0000","last_clean":"2024-09-17T22:59:20.866652+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T22:59:20.866652+0000","last_undegraded":"2024-09-17T22:59:20.866652+0000","last_fullsized":"2024-09-17T22:59:20.866652+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"130'365","reported_seq":18950,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:17.135736+0000","last_change":"2024-09-17T22:59:14.996187+0000","last_active":"2024-09-17T22:59:17.135736+0000","last_peered":"2024-09-17T22:59:17.135736+0000","last_clean":"2024-09-17T22:59:17.135736+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T22:59:17.135736+0000","last_undegraded":"2024-09-17T22:59:17.135736+0000","last_fullsized":"2024-09-17T22:59:17.135736+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:37.581054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045200999999999998,"stat_sum":{"num_bytes":28389176,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":19387,"num_read_kb":25736,"num_write":332,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.015914+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T22:59:18.015914+0000","last_peered":"2024-09-17T22:59:18.015914+0000","last_clean":"2024-09-17T22:59:18.015914+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T22:59:18.015914+0000","last_undegraded":"2024-09-17T22:59:18.015914+0000","last_fullsized":"2024-09-17T22:59:18.015914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"130'1215","reported_seq":12823,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.997984+0000","last_change":"2024-09-17T22:59:14.991025+0000","last_active":"2024-09-17T22:59:16.997984+0000","last_peered":"2024-09-17T22:59:16.997984+0000","last_clean":"2024-09-17T22:59:16.997984+0000","last_became_active":"2024-09-17T22:59:01.898399+0000","last_became_peered":"2024-09-17T22:59:01.898399+0000","last_unstale":"2024-09-17T22:59:16.997984+0000","last_undegraded":"2024-09-17T22:59:16.997984+0000","last_fullsized":"2024-09-17T22:59:16.997984+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1215,"log_dups_size":0,"ondisk_log_size":1215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:06.772814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044412599999999998,"stat_sum":{"num_bytes":17399392,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":16140,"num_read_kb":43091,"num_write":1871,"num_write_kb":4570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":674738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995493+0000","last_change":"2024-09-17T22:58:46.131382+0000","last_active":"2024-09-17T22:59:16.995493+0000","last_peered":"2024-09-17T22:59:16.995493+0000","last_clean":"2024-09-17T22:59:16.995493+0000","last_became_active":"2024-09-17T22:58:46.130955+0000","last_became_peered":"2024-09-17T22:58:46.130955+0000","last_unstale":"2024-09-17T22:59:16.995493+0000","last_undegraded":"2024-09-17T22:59:16.995493+0000","last_fullsized":"2024-09-17T22:59:16.995493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.813197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"121'815","reported_seq":6030,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994741+0000","last_change":"2024-09-17T22:59:14.991517+0000","last_active":"2024-09-17T22:59:16.994741+0000","last_peered":"2024-09-17T22:59:16.994741+0000","last_clean":"2024-09-17T22:59:16.994741+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T22:59:16.994741+0000","last_undegraded":"2024-09-17T22:59:16.994741+0000","last_fullsized":"2024-09-17T22:59:16.994741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:56:23.068486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001702505,"stat_sum":{"num_bytes":18865494,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":6951,"num_read_kb":15969,"num_write":1051,"num_write_kb":7549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"2.3","version":"19'1","reported_seq":75,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866696+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T22:59:20.866696+0000","last_peered":"2024-09-17T22:59:20.866696+0000","last_clean":"2024-09-17T22:59:20.866696+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T22:59:20.866696+0000","last_undegraded":"2024-09-17T22:59:20.866696+0000","last_fullsized":"2024-09-17T22:59:20.866696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.998202+0000","last_change":"2024-09-17T22:58:46.130584+0000","last_active":"2024-09-17T22:59:16.998202+0000","last_peered":"2024-09-17T22:59:16.998202+0000","last_clean":"2024-09-17T22:59:16.998202+0000","last_became_active":"2024-09-17T22:58:46.130202+0000","last_became_peered":"2024-09-17T22:58:46.130202+0000","last_unstale":"2024-09-17T22:59:16.998202+0000","last_undegraded":"2024-09-17T22:59:16.998202+0000","last_fullsized":"2024-09-17T22:59:16.998202+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:32.916711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"130'788","reported_seq":3735,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.995347+0000","last_change":"2024-09-17T22:59:14.995614+0000","last_active":"2024-09-17T22:59:16.995347+0000","last_peered":"2024-09-17T22:59:16.995347+0000","last_clean":"2024-09-17T22:59:16.995347+0000","last_became_active":"2024-09-17T22:59:01.896831+0000","last_became_peered":"2024-09-17T22:59:01.896831+0000","last_unstale":"2024-09-17T22:59:16.995347+0000","last_undegraded":"2024-09-17T22:59:16.995347+0000","last_fullsized":"2024-09-17T22:59:16.995347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:29.635477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079523300000000001,"stat_sum":{"num_bytes":42088635,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":5344,"num_read_kb":13669,"num_write":1097,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":3744943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.9","version":"130'505","reported_seq":3957,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.016920+0000","last_change":"2024-09-17T22:59:18.017416+0000","last_active":"2024-09-17T22:59:18.016920+0000","last_peered":"2024-09-17T22:59:18.016920+0000","last_clean":"2024-09-17T22:59:18.016920+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T22:59:18.016920+0000","last_undegraded":"2024-09-17T22:59:18.016920+0000","last_fullsized":"2024-09-17T22:59:18.016920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:55:46.203492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067207599999999997,"stat_sum":{"num_bytes":10471373,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5160,"num_read_kb":14284,"num_write":492,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.a","version":"131'526","reported_seq":3421,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.088546+0000","last_change":"2024-09-17T22:59:18.020579+0000","last_active":"2024-09-17T22:59:18.088546+0000","last_peered":"2024-09-17T22:59:18.088546+0000","last_clean":"2024-09-17T22:59:18.088546+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:59:18.088546+0000","last_undegraded":"2024-09-17T22:59:18.088546+0000","last_fullsized":"2024-09-17T22:59:18.088546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:08.003724+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003826075,"stat_sum":{"num_bytes":8015804,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":3596,"num_read_kb":19037,"num_write":619,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.b","version":"127'510","reported_seq":6142,"reported_epoch":130,"state":"active+undersized+remapped","last_fresh":"2024-09-17T22:59:17.186375+0000","last_change":"2024-09-17T22:59:17.186583+0000","last_active":"2024-09-17T22:59:17.186375+0000","last_peered":"2024-09-17T22:59:17.186375+0000","last_clean":"2024-09-17T22:59:14.059022+0000","last_became_active":"2024-09-17T22:59:17.018829+0000","last_became_peered":"2024-09-17T22:59:17.018829+0000","last_unstale":"2024-09-17T22:59:17.186375+0000","last_undegraded":"2024-09-17T22:59:17.186375+0000","last_fullsized":"2024-09-17T22:59:16.994496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090487300000000005,"stat_sum":{"num_bytes":15510395,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":6277,"num_read_kb":16384,"num_write":513,"num_write_kb":10978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","0","4"],"object_location_counts":[{"shards":"3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.c","version":"127'403","reported_seq":5020,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866755+0000","last_change":"2024-09-17T22:59:18.016581+0000","last_active":"2024-09-17T22:59:20.866755+0000","last_peered":"2024-09-17T22:59:20.866755+0000","last_clean":"2024-09-17T22:59:20.866755+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T22:59:20.866755+0000","last_undegraded":"2024-09-17T22:59:20.866755+0000","last_fullsized":"2024-09-17T22:59:20.866755+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:34.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652049999999999,"stat_sum":{"num_bytes":16549431,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":5438,"num_read_kb":18886,"num_write":470,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.d","version":"130'369","reported_seq":4573,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866910+0000","last_change":"2024-09-17T22:59:18.017682+0000","last_active":"2024-09-17T22:59:20.866910+0000","last_peered":"2024-09-17T22:59:20.866910+0000","last_clean":"2024-09-17T22:59:20.866910+0000","last_became_active":"2024-09-17T22:58:54.861414+0000","last_became_peered":"2024-09-17T22:58:54.861414+0000","last_unstale":"2024-09-17T22:59:20.866910+0000","last_undegraded":"2024-09-17T22:59:20.866910+0000","last_fullsized":"2024-09-17T22:59:20.866910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:14.726671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021113640000000001,"stat_sum":{"num_bytes":21422337,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5562,"num_read_kb":23049,"num_write":382,"num_write_kb":6583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.e","version":"127'323","reported_seq":1963,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994713+0000","last_change":"2024-09-17T22:59:14.995183+0000","last_active":"2024-09-17T22:59:16.994713+0000","last_peered":"2024-09-17T22:59:16.994713+0000","last_clean":"2024-09-17T22:59:16.994713+0000","last_became_active":"2024-09-17T22:59:01.896809+0000","last_became_peered":"2024-09-17T22:59:01.896809+0000","last_unstale":"2024-09-17T22:59:16.994713+0000","last_undegraded":"2024-09-17T22:59:16.994713+0000","last_fullsized":"2024-09-17T22:59:16.994713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:19:43.185212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060952000000000001,"stat_sum":{"num_bytes":12542734,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":1556,"num_read_kb":7770,"num_write":312,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2492972,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.f","version":"131'1021","reported_seq":8992,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.012564+0000","last_change":"2024-09-17T22:59:18.012564+0000","last_active":"2024-09-17T22:59:18.012564+0000","last_peered":"2024-09-17T22:59:18.012564+0000","last_clean":"2024-09-17T22:59:18.012564+0000","last_became_active":"2024-09-17T22:58:46.132674+0000","last_became_peered":"2024-09-17T22:58:46.132674+0000","last_unstale":"2024-09-17T22:59:18.012564+0000","last_undegraded":"2024-09-17T22:59:18.012564+0000","last_fullsized":"2024-09-17T22:59:18.012564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:52.449599+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0025620830000000002,"stat_sum":{"num_bytes":38757988,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":12100,"num_read_kb":34693,"num_write":1339,"num_write_kb":49269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.10","version":"127'644","reported_seq":5530,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-17T22:59:17.719756+0000","last_change":"2024-09-17T22:59:18.015342+0000","last_active":"2024-09-17T22:59:18.015342+0000","last_peered":"2024-09-17T22:59:17.719756+0000","last_clean":"2024-09-17T22:59:14.060717+0000","last_became_active":"2024-09-17T22:59:17.019737+0000","last_became_peered":"2024-09-17T22:59:17.019737+0000","last_unstale":"2024-09-17T22:59:17.719756+0000","last_undegraded":"2024-09-17T22:59:17.719756+0000","last_fullsized":"2024-09-17T22:59:16.998730+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048484400000000001,"stat_sum":{"num_bytes":65332590,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":5655,"num_read_kb":33513,"num_write":837,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.11","version":"131'490","reported_seq":4355,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867198+0000","last_change":"2024-09-17T22:59:18.016878+0000","last_active":"2024-09-17T22:59:20.867198+0000","last_peered":"2024-09-17T22:59:20.867198+0000","last_clean":"2024-09-17T22:59:20.867198+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T22:59:20.867198+0000","last_undegraded":"2024-09-17T22:59:20.867198+0000","last_fullsized":"2024-09-17T22:59:20.867198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:07.569383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647997,"stat_sum":{"num_bytes":13042207,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":4434,"num_read_kb":17010,"num_write":467,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.12","version":"120'753","reported_seq":13206,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.998060+0000","last_change":"2024-09-17T22:59:14.990249+0000","last_active":"2024-09-17T22:59:16.998060+0000","last_peered":"2024-09-17T22:59:16.998060+0000","last_clean":"2024-09-17T22:59:16.998060+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:59:16.998060+0000","last_undegraded":"2024-09-17T22:59:16.998060+0000","last_fullsized":"2024-09-17T22:59:16.998060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:22:24.860341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049187600000000003,"stat_sum":{"num_bytes":26054365,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":15096,"num_read_kb":33022,"num_write":952,"num_write_kb":9478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.13","version":"130'607","reported_seq":8883,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866147+0000","last_change":"2024-09-17T22:59:18.012873+0000","last_active":"2024-09-17T22:59:20.866147+0000","last_peered":"2024-09-17T22:59:20.866147+0000","last_clean":"2024-09-17T22:59:20.866147+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:59:20.866147+0000","last_undegraded":"2024-09-17T22:59:20.866147+0000","last_fullsized":"2024-09-17T22:59:20.866147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:19.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074441799999999995,"stat_sum":{"num_bytes":35205111,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":10373,"num_read_kb":26648,"num_write":676,"num_write_kb":10339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":944215323,"num_objects":1196,"num_object_clones":232,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":625,"num_read":254688,"num_read_kb":790090,"num_write":22190,"num_write_kb":365985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":105018507,"num_keys_recovered":514,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135028736,"data_stored":149542007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":19198,"ondisk_log_size":19198,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50668,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321172,"statfs":{"total":96636764160,"available":96584880128,"internally_reserved":0,"allocated":11427840,"data_stored":11394196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678032,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63612,"kb_used_data":22308,"kb_used_omap":17,"kb_used_meta":41262,"kb_avail":94308228,"statfs":{"total":96636764160,"available":96571625472,"internally_reserved":0,"allocated":22843392,"data_stored":23898875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":42252486},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90184,"kb_used_data":30568,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94281656,"statfs":{"total":96636764160,"available":96544415744,"internally_reserved":0,"allocated":31301632,"data_stored":34393366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74692,"kb_used_data":21284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297148,"statfs":{"total":96636764160,"available":96560279552,"internally_reserved":0,"allocated":21794816,"data_stored":23839469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977411,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":9348,"kb_used_omap":14,"kb_used_meta":28401,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":9572352,"data_stored":9610511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":29083050},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68044,"kb_used_data":18808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94303796,"statfs":{"total":96636764160,"available":96567087104,"internally_reserved":0,"allocated":19259392,"data_stored":19397216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62368,"kb_used_data":8968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309472,"statfs":{"total":96636764160,"available":96572899328,"internally_reserved":0,"allocated":9183232,"data_stored":8452450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84532,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287308,"statfs":{"total":96636764160,"available":96550203392,"internally_reserved":0,"allocated":27623424,"data_stored":30121396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7626752,"data_stored":7622593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15077376,"data_stored":16023503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":8856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18141184,"data_stored":20986702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29679616,"data_stored":33581478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21241856,"data_stored":23126463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9818112,"data_stored":10586377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25432064,"data_stored":28758635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:59:22.749 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T22:59:22.758 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:22.755+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:22.852 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:22.849+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:22.861 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:22.861 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:22.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:22.956 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:22.953+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:23.021 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.021 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.030 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:23.053 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:23.051+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:23.154 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:23.154 INFO:teuthology.orchestra.run.smithi159.stdout:{"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 138/2400 objects degraded (5.750%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":153,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1726613949,"num_in_osds":8,"osd_in_since":1726613931,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"peering","count":2},{"state_name":"active+undersized+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1200,"data_bytes":944805710,"bytes_used":544649216,"bytes_avail":772549464064,"bytes_total":773094113280,"inactive_pgs_ratio":0.048780485987663269,"degraded_objects":138,"degraded_total":2400,"degraded_ratio":0.057500000000000002,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":1715207,"recovering_keys_per_sec":0,"num_objects_recovered":134,"num_bytes_recovered":6862103,"num_keys_recovered":0,"read_bytes_sec":88303,"write_bytes_sec":5375,"read_op_per_sec":97,"write_op_per_sec":15},"fsmap":{"epoch":1,"btime":"2024-09-17T22:56:50:048469+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T22:59:12.212978+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T22:59:23.154 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:23.152+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:23.167 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-17T22:59:23.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:59:23.174 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.174 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.183 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:23.255 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:23.252+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:23.288 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.288 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.297 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:23.356 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:23.353+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:23.403 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.403 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.412 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:23.456 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:23.453+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:23.493 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:23.494 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:59:23.506 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-17T22:59:22.863493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":919378643,"num_objects":1182,"num_object_clones":226,"num_object_copies":2364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1182,"num_whiteouts":620,"num_read":254973,"num_read_kb":790328,"num_write":22390,"num_write_kb":368418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3738,"num_bytes_recovered":118310249,"num_keys_recovered":546,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19365,"ondisk_log_size":19365,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":548800,"kb_used_data":158020,"kb_used_omap":41,"kb_used_meta":388502,"kb_avail":754425920,"statfs":{"total":773094113280,"available":772532142080,"internally_reserved":0,"allocated":161812480,"data_stored":169530962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":397826329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30810870,"num_objects":-24,"num_object_clones":-6,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":-5,"num_read":348,"num_read_kb":295,"num_write":65,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":676,"num_bytes_recovered":13291704,"num_keys_recovered":32,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.837576"},"pg_stats":[{"pgid":"3.1f","version":"127'440","reported_seq":4583,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865378+0000","last_change":"2024-09-17T22:59:19.038169+0000","last_active":"2024-09-17T22:59:21.865378+0000","last_peered":"2024-09-17T22:59:21.865378+0000","last_clean":"2024-09-17T22:59:21.865378+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T22:59:21.865378+0000","last_undegraded":"2024-09-17T22:59:21.865378+0000","last_fullsized":"2024-09-17T22:59:21.865378+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:01.405536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178276,"stat_sum":{"num_bytes":87780822,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":4242,"num_read_kb":23843,"num_write":442,"num_write_kb":24358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1e","version":"130'298","reported_seq":3252,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868133+0000","last_change":"2024-09-17T22:59:18.019203+0000","last_active":"2024-09-17T22:59:21.868133+0000","last_peered":"2024-09-17T22:59:21.868133+0000","last_clean":"2024-09-17T22:59:21.868133+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:59:21.868133+0000","last_undegraded":"2024-09-17T22:59:21.868133+0000","last_fullsized":"2024-09-17T22:59:21.868133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:02.822711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345557,"stat_sum":{"num_bytes":25582696,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2647,"num_read_kb":17542,"num_write":273,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1d","version":"120'534","reported_seq":9694,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867018+0000","last_change":"2024-09-17T22:59:18.017802+0000","last_active":"2024-09-17T22:59:20.867018+0000","last_peered":"2024-09-17T22:59:20.867018+0000","last_clean":"2024-09-17T22:59:20.867018+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T22:59:20.867018+0000","last_undegraded":"2024-09-17T22:59:20.867018+0000","last_fullsized":"2024-09-17T22:59:20.867018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:17:11.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002212122,"stat_sum":{"num_bytes":21602335,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":10247,"num_read_kb":31376,"num_write":558,"num_write_kb":6655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1c","version":"131'539","reported_seq":5651,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866792+0000","last_change":"2024-09-17T22:59:18.018154+0000","last_active":"2024-09-17T22:59:20.866792+0000","last_peered":"2024-09-17T22:59:20.866792+0000","last_clean":"2024-09-17T22:59:20.866792+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T22:59:20.866792+0000","last_undegraded":"2024-09-17T22:59:20.866792+0000","last_fullsized":"2024-09-17T22:59:20.866792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:48.792497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025587560000000001,"stat_sum":{"num_bytes":18517824,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6303,"num_read_kb":16894,"num_write":562,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1b","version":"135'701","reported_seq":14870,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.875255+0000","last_change":"2024-09-17T22:59:19.039165+0000","last_active":"2024-09-17T22:59:21.875255+0000","last_peered":"2024-09-17T22:59:21.875255+0000","last_clean":"2024-09-17T22:59:21.875255+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T22:59:21.875255+0000","last_undegraded":"2024-09-17T22:59:21.875255+0000","last_fullsized":"2024-09-17T22:59:21.875255+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:42.940595+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0044050690000000002,"stat_sum":{"num_bytes":19229053,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":16573,"num_read_kb":56613,"num_write":722,"num_write_kb":16994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1a","version":"130'404","reported_seq":10158,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.054935+0000","last_change":"2024-09-17T22:59:18.010632+0000","last_active":"2024-09-17T22:59:18.054935+0000","last_peered":"2024-09-17T22:59:18.054935+0000","last_clean":"2024-09-17T22:59:18.054935+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:59:18.054935+0000","last_undegraded":"2024-09-17T22:59:18.054935+0000","last_fullsized":"2024-09-17T22:59:18.054935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:42.864990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072336099999999995,"stat_sum":{"num_bytes":21547601,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":10456,"num_read_kb":23168,"num_write":395,"num_write_kb":11995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.19","version":"127'739","reported_seq":5439,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868938+0000","last_change":"2024-09-17T22:59:18.015821+0000","last_active":"2024-09-17T22:59:21.868938+0000","last_peered":"2024-09-17T22:59:21.868938+0000","last_clean":"2024-09-17T22:59:21.868938+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:59:21.868938+0000","last_undegraded":"2024-09-17T22:59:21.868938+0000","last_fullsized":"2024-09-17T22:59:21.868938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:26.149566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066901299999999999,"stat_sum":{"num_bytes":44877072,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":6796,"num_read_kb":19698,"num_write":848,"num_write_kb":4687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.18","version":"132'476","reported_seq":5715,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868149+0000","last_change":"2024-09-17T22:59:19.041565+0000","last_active":"2024-09-17T22:59:21.868149+0000","last_peered":"2024-09-17T22:59:21.868149+0000","last_clean":"2024-09-17T22:59:21.868149+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T22:59:21.868149+0000","last_undegraded":"2024-09-17T22:59:21.868149+0000","last_fullsized":"2024-09-17T22:59:21.868149+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:25:44.508190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0054511580000000002,"stat_sum":{"num_bytes":32039559,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":4737,"num_read_kb":27532,"num_write":456,"num_write_kb":27831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.17","version":"132'674","reported_seq":4543,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865435+0000","last_change":"2024-09-17T22:59:18.025366+0000","last_active":"2024-09-17T22:59:21.865435+0000","last_peered":"2024-09-17T22:59:21.865435+0000","last_clean":"2024-09-17T22:59:21.865435+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T22:59:21.865435+0000","last_undegraded":"2024-09-17T22:59:21.865435+0000","last_fullsized":"2024-09-17T22:59:21.865435+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:21:55.359946+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055098029999999997,"stat_sum":{"num_bytes":32932558,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":5593,"num_read_kb":21131,"num_write":811,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.16","version":"127'1303","reported_seq":9540,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866783+0000","last_change":"2024-09-17T22:59:18.017250+0000","last_active":"2024-09-17T22:59:20.866783+0000","last_peered":"2024-09-17T22:59:20.866783+0000","last_clean":"2024-09-17T22:59:20.866783+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T22:59:20.866783+0000","last_undegraded":"2024-09-17T22:59:20.866783+0000","last_fullsized":"2024-09-17T22:59:20.866783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1303,"log_dups_size":0,"ondisk_log_size":1303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:39.632576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695525,"stat_sum":{"num_bytes":40767942,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":13640,"num_read_kb":44259,"num_write":1892,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.15","version":"127'433","reported_seq":4701,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868164+0000","last_change":"2024-09-17T22:59:18.024738+0000","last_active":"2024-09-17T22:59:21.868164+0000","last_peered":"2024-09-17T22:59:21.868164+0000","last_clean":"2024-09-17T22:59:21.868164+0000","last_became_active":"2024-09-17T22:58:46.132716+0000","last_became_peered":"2024-09-17T22:58:46.132716+0000","last_unstale":"2024-09-17T22:59:21.868164+0000","last_undegraded":"2024-09-17T22:59:21.868164+0000","last_fullsized":"2024-09-17T22:59:21.868164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:34.474667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061835429999999997,"stat_sum":{"num_bytes":63637947,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":4517,"num_read_kb":28303,"num_write":404,"num_write_kb":9577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.14","version":"130'462","reported_seq":2769,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.010695+0000","last_change":"2024-09-17T22:59:18.011299+0000","last_active":"2024-09-17T22:59:18.010695+0000","last_peered":"2024-09-17T22:59:18.010695+0000","last_clean":"2024-09-17T22:59:18.010695+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:59:18.010695+0000","last_undegraded":"2024-09-17T22:59:18.010695+0000","last_fullsized":"2024-09-17T22:59:18.010695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:12.990255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689870000000001,"stat_sum":{"num_bytes":15554461,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":3496,"num_read_kb":11431,"num_write":451,"num_write_kb":5675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.014877+0000","last_change":"2024-09-17T22:59:11.956325+0000","last_active":"2024-09-17T22:59:18.014877+0000","last_peered":"2024-09-17T22:59:18.014877+0000","last_clean":"2024-09-17T22:59:18.014877+0000","last_became_active":"2024-09-17T22:59:11.956025+0000","last_became_peered":"2024-09-17T22:59:11.956025+0000","last_unstale":"2024-09-17T22:59:18.014877+0000","last_undegraded":"2024-09-17T22:59:18.014877+0000","last_fullsized":"2024-09-17T22:59:18.014877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:21:59.355000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"132'602","reported_seq":10006,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865400+0000","last_change":"2024-09-17T22:59:18.020410+0000","last_active":"2024-09-17T22:59:21.865400+0000","last_peered":"2024-09-17T22:59:21.865400+0000","last_clean":"2024-09-17T22:59:21.865400+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:59:21.865400+0000","last_undegraded":"2024-09-17T22:59:21.865400+0000","last_fullsized":"2024-09-17T22:59:21.865400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:10.984755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010349560000000001,"stat_sum":{"num_bytes":47054077,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":11985,"num_read_kb":26141,"num_write":527,"num_write_kb":10122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869786+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:59:21.869786+0000","last_peered":"2024-09-17T22:59:21.869786+0000","last_clean":"2024-09-17T22:59:21.869786+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:59:21.869786+0000","last_undegraded":"2024-09-17T22:59:21.869786+0000","last_fullsized":"2024-09-17T22:59:21.869786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"127'483","reported_seq":9581,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868645+0000","last_change":"2024-09-17T22:59:18.018460+0000","last_active":"2024-09-17T22:59:21.868645+0000","last_peered":"2024-09-17T22:59:21.868645+0000","last_clean":"2024-09-17T22:59:21.868645+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:59:21.868645+0000","last_undegraded":"2024-09-17T22:59:21.868645+0000","last_fullsized":"2024-09-17T22:59:21.868645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:19.050560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060727000000000003,"stat_sum":{"num_bytes":12802556,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":9662,"num_read_kb":29730,"num_write":502,"num_write_kb":5475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866222+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:59:20.866222+0000","last_peered":"2024-09-17T22:59:20.866222+0000","last_clean":"2024-09-17T22:59:20.866222+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:59:20.866222+0000","last_undegraded":"2024-09-17T22:59:20.866222+0000","last_fullsized":"2024-09-17T22:59:20.866222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"130'509","reported_seq":9608,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869317+0000","last_change":"2024-09-17T22:59:18.016548+0000","last_active":"2024-09-17T22:59:21.869317+0000","last_peered":"2024-09-17T22:59:21.869317+0000","last_clean":"2024-09-17T22:59:21.869317+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T22:59:21.869317+0000","last_undegraded":"2024-09-17T22:59:21.869317+0000","last_fullsized":"2024-09-17T22:59:21.869317+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:19.043740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013874099999999999,"stat_sum":{"num_bytes":10497666,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":10330,"num_read_kb":26999,"num_write":488,"num_write_kb":10537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869043+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:59:21.869043+0000","last_peered":"2024-09-17T22:59:21.869043+0000","last_clean":"2024-09-17T22:59:21.869043+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:59:21.869043+0000","last_undegraded":"2024-09-17T22:59:21.869043+0000","last_fullsized":"2024-09-17T22:59:21.869043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"130'703","reported_seq":3755,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866901+0000","last_change":"2024-09-17T22:59:18.016216+0000","last_active":"2024-09-17T22:59:20.866901+0000","last_peered":"2024-09-17T22:59:20.866901+0000","last_clean":"2024-09-17T22:59:20.866901+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T22:59:20.866901+0000","last_undegraded":"2024-09-17T22:59:20.866901+0000","last_fullsized":"2024-09-17T22:59:20.866901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:38.503327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091405800000000004,"stat_sum":{"num_bytes":16532738,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":5442,"num_read_kb":19069,"num_write":877,"num_write_kb":12167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.3","version":"131'605","reported_seq":4964,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.073061+0000","last_change":"2024-09-17T22:59:18.012038+0000","last_active":"2024-09-17T22:59:18.073061+0000","last_peered":"2024-09-17T22:59:18.073061+0000","last_clean":"2024-09-17T22:59:18.073061+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:59:18.073061+0000","last_undegraded":"2024-09-17T22:59:18.073061+0000","last_fullsized":"2024-09-17T22:59:18.073061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:12.453871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001499923,"stat_sum":{"num_bytes":42378885,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":5114,"num_read_kb":23733,"num_write":607,"num_write_kb":9693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866652+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T22:59:20.866652+0000","last_peered":"2024-09-17T22:59:20.866652+0000","last_clean":"2024-09-17T22:59:20.866652+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T22:59:20.866652+0000","last_undegraded":"2024-09-17T22:59:20.866652+0000","last_fullsized":"2024-09-17T22:59:20.866652+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"130'365","reported_seq":18966,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869191+0000","last_change":"2024-09-17T22:59:18.017081+0000","last_active":"2024-09-17T22:59:21.869191+0000","last_peered":"2024-09-17T22:59:21.869191+0000","last_clean":"2024-09-17T22:59:21.869191+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T22:59:21.869191+0000","last_undegraded":"2024-09-17T22:59:21.869191+0000","last_fullsized":"2024-09-17T22:59:21.869191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:25:26.049783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001445574,"stat_sum":{"num_bytes":28389176,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":19387,"num_read_kb":25736,"num_write":332,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.015914+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T22:59:18.015914+0000","last_peered":"2024-09-17T22:59:18.015914+0000","last_clean":"2024-09-17T22:59:18.015914+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T22:59:18.015914+0000","last_undegraded":"2024-09-17T22:59:18.015914+0000","last_fullsized":"2024-09-17T22:59:18.015914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'1217","reported_seq":13081,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.872814+0000","last_change":"2024-09-17T22:59:21.871845+0000","last_active":"2024-09-17T22:59:21.872814+0000","last_peered":"2024-09-17T22:59:21.872814+0000","last_clean":"2024-09-17T22:59:21.872814+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T22:59:21.872814+0000","last_undegraded":"2024-09-17T22:59:21.872814+0000","last_fullsized":"2024-09-17T22:59:21.872814+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:12:13.173957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064974900000000003,"stat_sum":{"num_bytes":17399392,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":16146,"num_read_kb":43095,"num_write":1873,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869274+0000","last_change":"2024-09-17T22:58:46.131382+0000","last_active":"2024-09-17T22:59:21.869274+0000","last_peered":"2024-09-17T22:59:21.869274+0000","last_clean":"2024-09-17T22:59:21.869274+0000","last_became_active":"2024-09-17T22:58:46.130955+0000","last_became_peered":"2024-09-17T22:58:46.130955+0000","last_unstale":"2024-09-17T22:59:21.869274+0000","last_undegraded":"2024-09-17T22:59:21.869274+0000","last_fullsized":"2024-09-17T22:59:21.869274+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.813197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"131'818","reported_seq":6048,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868205+0000","last_change":"2024-09-17T22:59:18.025680+0000","last_active":"2024-09-17T22:59:21.868205+0000","last_peered":"2024-09-17T22:59:21.868205+0000","last_clean":"2024-09-17T22:59:21.868205+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T22:59:21.868205+0000","last_undegraded":"2024-09-17T22:59:21.868205+0000","last_fullsized":"2024-09-17T22:59:21.868205+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:49.133668+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0076769289999999999,"stat_sum":{"num_bytes":14671190,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":6951,"num_read_kb":15969,"num_write":1052,"num_write_kb":7549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.3","version":"19'1","reported_seq":75,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866696+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T22:59:20.866696+0000","last_peered":"2024-09-17T22:59:20.866696+0000","last_clean":"2024-09-17T22:59:20.866696+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T22:59:20.866696+0000","last_undegraded":"2024-09-17T22:59:20.866696+0000","last_fullsized":"2024-09-17T22:59:20.866696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865206+0000","last_change":"2024-09-17T22:58:46.130584+0000","last_active":"2024-09-17T22:59:21.865206+0000","last_peered":"2024-09-17T22:59:21.865206+0000","last_clean":"2024-09-17T22:59:21.865206+0000","last_became_active":"2024-09-17T22:58:46.130202+0000","last_became_peered":"2024-09-17T22:58:46.130202+0000","last_unstale":"2024-09-17T22:59:21.865206+0000","last_undegraded":"2024-09-17T22:59:21.865206+0000","last_fullsized":"2024-09-17T22:59:21.865206+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:32.916711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"130'788","reported_seq":3752,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.874714+0000","last_change":"2024-09-17T22:59:21.872737+0000","last_active":"2024-09-17T22:59:21.874714+0000","last_peered":"2024-09-17T22:59:21.874714+0000","last_clean":"2024-09-17T22:59:21.874714+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T22:59:21.874714+0000","last_undegraded":"2024-09-17T22:59:21.874714+0000","last_fullsized":"2024-09-17T22:59:21.874714+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:00.316496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050777699999999995,"stat_sum":{"num_bytes":42088635,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":5344,"num_read_kb":13669,"num_write":1097,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.9","version":"130'505","reported_seq":3957,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.016920+0000","last_change":"2024-09-17T22:59:18.017416+0000","last_active":"2024-09-17T22:59:18.016920+0000","last_peered":"2024-09-17T22:59:18.016920+0000","last_clean":"2024-09-17T22:59:18.016920+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T22:59:18.016920+0000","last_undegraded":"2024-09-17T22:59:18.016920+0000","last_fullsized":"2024-09-17T22:59:18.016920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:55:46.203492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067207599999999997,"stat_sum":{"num_bytes":10471373,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5160,"num_read_kb":14284,"num_write":492,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.a","version":"131'526","reported_seq":3421,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.088546+0000","last_change":"2024-09-17T22:59:18.020579+0000","last_active":"2024-09-17T22:59:18.088546+0000","last_peered":"2024-09-17T22:59:18.088546+0000","last_clean":"2024-09-17T22:59:18.088546+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:59:18.088546+0000","last_undegraded":"2024-09-17T22:59:18.088546+0000","last_fullsized":"2024-09-17T22:59:18.088546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:08.003724+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003826075,"stat_sum":{"num_bytes":8015804,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":3596,"num_read_kb":19037,"num_write":619,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.b","version":"135'511","reported_seq":6167,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.875674+0000","last_change":"2024-09-17T22:59:19.034403+0000","last_active":"2024-09-17T22:59:21.875674+0000","last_peered":"2024-09-17T22:59:21.875674+0000","last_clean":"2024-09-17T22:59:21.875674+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T22:59:21.875674+0000","last_undegraded":"2024-09-17T22:59:21.875674+0000","last_fullsized":"2024-09-17T22:59:21.875674+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:27.626258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011073000000000001,"stat_sum":{"num_bytes":15510131,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":6279,"num_read_kb":16385,"num_write":514,"num_write_kb":10978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.c","version":"127'403","reported_seq":5020,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866755+0000","last_change":"2024-09-17T22:59:18.016581+0000","last_active":"2024-09-17T22:59:20.866755+0000","last_peered":"2024-09-17T22:59:20.866755+0000","last_clean":"2024-09-17T22:59:20.866755+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T22:59:20.866755+0000","last_undegraded":"2024-09-17T22:59:20.866755+0000","last_fullsized":"2024-09-17T22:59:20.866755+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:34.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652049999999999,"stat_sum":{"num_bytes":16549431,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":5438,"num_read_kb":18886,"num_write":470,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.d","version":"130'369","reported_seq":4573,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866910+0000","last_change":"2024-09-17T22:59:18.017682+0000","last_active":"2024-09-17T22:59:20.866910+0000","last_peered":"2024-09-17T22:59:20.866910+0000","last_clean":"2024-09-17T22:59:20.866910+0000","last_became_active":"2024-09-17T22:58:54.861414+0000","last_became_peered":"2024-09-17T22:58:54.861414+0000","last_unstale":"2024-09-17T22:59:20.866910+0000","last_undegraded":"2024-09-17T22:59:20.866910+0000","last_fullsized":"2024-09-17T22:59:20.866910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:14.726671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021113640000000001,"stat_sum":{"num_bytes":21422337,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5562,"num_read_kb":23049,"num_write":382,"num_write_kb":6583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.e","version":"127'323","reported_seq":1963,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T22:59:16.994713+0000","last_change":"2024-09-17T22:59:14.995183+0000","last_active":"2024-09-17T22:59:16.994713+0000","last_peered":"2024-09-17T22:59:16.994713+0000","last_clean":"2024-09-17T22:59:16.994713+0000","last_became_active":"2024-09-17T22:59:01.896809+0000","last_became_peered":"2024-09-17T22:59:01.896809+0000","last_unstale":"2024-09-17T22:59:16.994713+0000","last_undegraded":"2024-09-17T22:59:16.994713+0000","last_fullsized":"2024-09-17T22:59:16.994713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:19:43.185212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060952000000000001,"stat_sum":{"num_bytes":12542734,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":1556,"num_read_kb":7770,"num_write":312,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2492972,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"19"}]},{"pgid":"3.f","version":"131'1021","reported_seq":8992,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T22:59:18.012564+0000","last_change":"2024-09-17T22:59:18.012564+0000","last_active":"2024-09-17T22:59:18.012564+0000","last_peered":"2024-09-17T22:59:18.012564+0000","last_clean":"2024-09-17T22:59:18.012564+0000","last_became_active":"2024-09-17T22:58:46.132674+0000","last_became_peered":"2024-09-17T22:58:46.132674+0000","last_unstale":"2024-09-17T22:59:18.012564+0000","last_undegraded":"2024-09-17T22:59:18.012564+0000","last_fullsized":"2024-09-17T22:59:18.012564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:52.449599+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0025620830000000002,"stat_sum":{"num_bytes":38757988,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":12100,"num_read_kb":34693,"num_write":1339,"num_write_kb":49269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"1a"}]},{"pgid":"3.10","version":"127'644","reported_seq":5530,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-17T22:59:17.719756+0000","last_change":"2024-09-17T22:59:18.015342+0000","last_active":"2024-09-17T22:59:18.015342+0000","last_peered":"2024-09-17T22:59:17.719756+0000","last_clean":"2024-09-17T22:59:14.060717+0000","last_became_active":"2024-09-17T22:59:17.019737+0000","last_became_peered":"2024-09-17T22:59:17.019737+0000","last_unstale":"2024-09-17T22:59:17.719756+0000","last_undegraded":"2024-09-17T22:59:17.719756+0000","last_fullsized":"2024-09-17T22:59:16.998730+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048484400000000001,"stat_sum":{"num_bytes":65332590,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":5655,"num_read_kb":33513,"num_write":837,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1f","length":"18"}]},{"pgid":"3.11","version":"131'490","reported_seq":4355,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867198+0000","last_change":"2024-09-17T22:59:18.016878+0000","last_active":"2024-09-17T22:59:20.867198+0000","last_peered":"2024-09-17T22:59:20.867198+0000","last_clean":"2024-09-17T22:59:20.867198+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T22:59:20.867198+0000","last_undegraded":"2024-09-17T22:59:20.867198+0000","last_fullsized":"2024-09-17T22:59:20.867198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:07.569383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647997,"stat_sum":{"num_bytes":13042207,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":4434,"num_read_kb":17010,"num_write":467,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.12","version":"120'753","reported_seq":13221,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865204+0000","last_change":"2024-09-17T22:59:18.021520+0000","last_active":"2024-09-17T22:59:21.865204+0000","last_peered":"2024-09-17T22:59:21.865204+0000","last_clean":"2024-09-17T22:59:21.865204+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:59:21.865204+0000","last_undegraded":"2024-09-17T22:59:21.865204+0000","last_fullsized":"2024-09-17T22:59:21.865204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:31.309813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025369390000000002,"stat_sum":{"num_bytes":26054365,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":15096,"num_read_kb":33022,"num_write":952,"num_write_kb":9478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.13","version":"130'607","reported_seq":8883,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866147+0000","last_change":"2024-09-17T22:59:18.012873+0000","last_active":"2024-09-17T22:59:20.866147+0000","last_peered":"2024-09-17T22:59:20.866147+0000","last_clean":"2024-09-17T22:59:20.866147+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:59:20.866147+0000","last_undegraded":"2024-09-17T22:59:20.866147+0000","last_fullsized":"2024-09-17T22:59:20.866147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:19.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074441799999999995,"stat_sum":{"num_bytes":35205111,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":10373,"num_read_kb":26648,"num_write":676,"num_write_kb":10339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":918788256,"num_objects":1178,"num_object_clones":226,"num_object_copies":2356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1178,"num_whiteouts":620,"num_read":254857,"num_read_kb":790228,"num_write":22229,"num_write_kb":366000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3735,"num_bytes_recovered":118310211,"num_keys_recovered":546,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147828736,"data_stored":162113733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":19245,"ondisk_log_size":19245,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50668,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321172,"statfs":{"total":96636764160,"available":96584880128,"internally_reserved":0,"allocated":11427840,"data_stored":11394196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678032,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63612,"kb_used_data":22308,"kb_used_omap":17,"kb_used_meta":41262,"kb_avail":94308228,"statfs":{"total":96636764160,"available":96571625472,"internally_reserved":0,"allocated":22843392,"data_stored":23898875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":42252486},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90184,"kb_used_data":30568,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94281656,"statfs":{"total":96636764160,"available":96544415744,"internally_reserved":0,"allocated":31301632,"data_stored":34393366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75536,"kb_used_data":17976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94296304,"statfs":{"total":96636764160,"available":96559415296,"internally_reserved":0,"allocated":18407424,"data_stored":18579293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977411,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":9348,"kb_used_omap":14,"kb_used_meta":28401,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":9572352,"data_stored":9610511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":29083050},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":13372,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":13692928,"data_stored":13641694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83868,"kb_used_data":26312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287972,"statfs":{"total":96636764160,"available":96550883328,"internally_reserved":0,"allocated":26943488,"data_stored":27891631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84532,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287308,"statfs":{"total":96636764160,"available":96550203392,"internally_reserved":0,"allocated":27623424,"data_stored":30121396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24748032,"data_stored":26528606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12091392,"data_stored":12869282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":8856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16805888,"data_stored":17806636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29679616,"data_stored":33581478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21241856,"data_stored":23126463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9818112,"data_stored":10586377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25432064,"data_stored":28758635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:59:23.557 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.558 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.558 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:23.554+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:23.566 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:23.659 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:23.656+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:23.670 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.670 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.679 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:23.763 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:23.761+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:23.822 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.823 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.831 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:23.860 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:23.857+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:23.961 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:23.958+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:23.976 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:59:23.987 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:23.987 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:23.997 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:24.082 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:24.080+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:24.144 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:24.144 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:24.153 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:24.163 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:24.160+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:24.263 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:24.261+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:24.298 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:24.298 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:24.306 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:24.365 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:24.362+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:24.451 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:24.451 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:24.460 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:24.466 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:24.463+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:24.562 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:24.562 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:24.566 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:24.564+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:24.570 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:24.667 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:24.665+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:24.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:24.715 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:24.715 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:24.725 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:24.769 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:24.766+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:24.821 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:24.821 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:24.838 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:24.838 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:24.855 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:24.855 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:24.870 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:24.867+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:24.871 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:24.874 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:24.874 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:24.880 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:24.880 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:24.880 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:24.881 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:24.881 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:24.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:24.894 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:24.895 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:24.915 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:24.915 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:24.933 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:24.933 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:24.952 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:24.952 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:25.004 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:25.002+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 85.854667847000002, 2024-09-17T22:59:25.011 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:59:25.013 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:59:25.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:59:25.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:59:25.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.017 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502879448000002, 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:59:25.018 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.019 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.020 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:59:25.021 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:59:25.022 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502858738, 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.023 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.024 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:59:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:59:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502838943, 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.027 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.028 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.029 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:59:25.030 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.031 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502819700000003, 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.032 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:25.033 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:59:25.034 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:59:25.035 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502798560000002, 2024-09-17T22:59:25.036 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.037 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:25.038 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.039 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.040 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:59:25.041 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502773470999998, 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.042 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.043 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:59:25.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.045 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502754128000007, 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.047 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:59:25.048 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:59:25.049 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502732105000007, 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:59:25.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.053 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502637964000002, 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.055 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.056 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:59:25.057 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502611192000003, 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.059 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.060 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.061 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.062 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502590666000003, 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.063 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.064 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.065 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502569373, 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.067 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.068 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.069 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.070 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502540498000002, 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:59:25.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:25.073 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.074 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.075 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502499419000003, 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.076 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.078 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:59:25.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 71.502475395999994, 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:59:25.080 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.081 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.082 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.083 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.617732769, 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:59:25.084 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.085 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:59:25.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.087 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.088 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.617711141000001, 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.089 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:59:25.090 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.091 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.092 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.617689956999996, 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.093 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:59:25.095 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.096 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:25.097 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 67.617668877, 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:25.098 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:25.099 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.100 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:59:25.101 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.102 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.102 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.102 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:25.102 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:25.102 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:25.103 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:25.104 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:25.101+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:25.204 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:25.202+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:25.207 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:25.208 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:25.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:25.305 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:25.303+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:25.320 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:25.320 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:25.320 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:25.329 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:25.406 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:25.403+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.045687560999994, 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.006714608999999, 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.006693833, 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.006675043000001, 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.006629832000002, 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.006596004000002, 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301678737000003, 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301658269000001, 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:25.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:25.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301636707999997, 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:25.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:25.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301616891000002, 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.482 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:25.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:25.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:25.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.487 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.301596902999997, 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.488 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:25.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:25.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300931490000004, 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:25.493 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:25.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:25.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.498 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.499 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300908745000001, 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.500 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:25.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:25.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.505 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300887611999997, 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:25.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:25.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.510 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300867494999999, 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.511 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300840285, 2024-09-17T22:59:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.519 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:25.520 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:25.521 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300669423999999, 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.523 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:25.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.525 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.526 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.300645105999997, 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.527 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.528 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.529 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.529 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:25.529 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.529 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.529 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.530 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:25.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:25.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:25.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:25.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:25.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:25.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.536 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.295371103999997, 2024-09-17T22:59:25.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.538 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:25.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:25.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:25.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.543 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:25.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:25.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:25.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:25.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:25.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.548 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:25.548 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:25.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:25.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:25.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:25.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:25.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200002 3.1b 3:dea2114f:::rbd_object_map.11884120dce1:head [call lock.unlock in=10b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.4367966580000004, 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0273163620000001, 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:25.553 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200002 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:25.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994166+0000", 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994172+0000", 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2330000000000003e-06 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994181+0000", 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3190000000000001e-06 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994188+0000", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8190000000000003e-06 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994239+0000", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0865000000000002e-05 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994250+0000", 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1191e-05 2024-09-17T22:59:25.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019705+0000", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0254550520000001 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019922+0000", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021627500000000001 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020180+0000", 2024-09-17T22:59:25.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025870299999999999 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020983+0000", 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080232199999999995 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021341+0000", 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000358356 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021391+0000", 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0390999999999999e-05 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021481+0000", 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9673999999999996e-05 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.561 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.562 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:25.562 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:25.562 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:25.563 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:25.564 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:25.560+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:25.664 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:25.661+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:25.708 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:25.708 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:25.708 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:25.717 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:25.765 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:25.763+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:25.860 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:25.860 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:25.861 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:25.861 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:25.866 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:25.863+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:25.869 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:25.966 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:25.964+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.761336505, 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:26.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.7613205, 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.022 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:26.024 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.026 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.76128773000001, 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.027 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:26.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:26.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.031 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.76125999200001, 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.032 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:26.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.761237573, 2024-09-17T22:59:26.036 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:26.037 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.038 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.039 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.040 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.76121529300001, 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.041 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.042 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.043 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.044 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.045 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.76100788799999, 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.046 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.047 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.048 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.049 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.760461273, 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:26.050 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.051 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:26.052 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.75939007700001, 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.759378622, 2024-09-17T22:59:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.065 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.759367645, 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.066 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.067 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:26.068 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.069 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.75729351299999, 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.070 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.071 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.072 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:26.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.075 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.757281199, 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.757267962, 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.080 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:26.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.082 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.083 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.084 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.757257133, 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:26.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:26.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.755298258, 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.089 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:26.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:26.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:26.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.75527390000001, 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.093 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:26.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.755244682, 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:26.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:26.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 109.754465012, 2024-09-17T22:59:26.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:26.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.106 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.135606473999999, 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.107 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:26.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:26.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:26.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.114 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.116 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:26.116 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:26.113+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:26.216 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:26.212+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:26.221 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.221 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:26.221 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:26.222 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.231 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:26.317 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:26.313+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:26.375 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.375 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:26.376 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:26.376 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:26.376 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.384 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:26.417 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:26.415+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:26.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:26.530 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 142.66606224099999, 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.531 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:26.532 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.533 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:26.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.535 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:26.536 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.537 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.439223114, 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.538 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.539 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.540 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:26.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:26.542 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.439204588, 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.543 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.544 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.545 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.439186416, 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.548 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.550 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.551 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:26.552 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:26.553 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.439168588, 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.554 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.555 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.556 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.557 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:26.558 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 115.438853455, 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.559 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:26.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.561 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.562 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.563 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.957029120999998, 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.564 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.565 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:26.566 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.567 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:26.568 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.957019492999997, 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.569 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.570 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:26.571 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:26.572 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.573 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.574 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.957010144999998, 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.575 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.576 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:26.577 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.578 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.579 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.957000223000001, 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.580 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.581 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:26.582 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.584 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.585 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956986766999997, 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.586 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.587 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.588 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:26.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.590 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:26.591 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956976955000002, 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:26.592 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:26.594 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.595 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:26.596 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956946586000001, 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.597 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.598 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.599 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.600 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:26.601 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.602 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956934263000001, 2024-09-17T22:59:26.603 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.604 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.605 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.607 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.608 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.609 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.610 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:26.611 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:26.612 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.613 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956924755999999, 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.614 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.615 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.616 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.617 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.618 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.619 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:26.620 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:26.621 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:26.622 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956914715000003, 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.624 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.625 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:26.626 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.628 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956905431000003, 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.629 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.630 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:26.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.632 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.633 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956895936999999, 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.634 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:26.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.636 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:26.637 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:26.638 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956884668999997, 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.639 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.640 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:26.641 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.642 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.956875263999997, 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:26.643 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:26.645 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.646 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:26.647 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:26.648 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.649 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:26.552+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:26.649 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:26.647+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:26.650 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:26.750 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:26.747+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:26.753 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.753 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:26.753 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:26.753 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.762 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:26.787 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:59:26.800 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-17T22:59:26.215328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918331955,"num_objects":1174,"num_object_clones":226,"num_object_copies":2348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1174,"num_whiteouts":620,"num_read":254991,"num_read_kb":790344,"num_write":22403,"num_write_kb":368421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19378,"ondisk_log_size":19378,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":531480,"kb_used_data":155220,"kb_used_omap":41,"kb_used_meta":376022,"kb_avail":754443240,"statfs":{"total":773094113280,"available":772549877760,"internally_reserved":0,"allocated":158945280,"data_stored":166230022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":385046809},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26473755,"num_objects":-26,"num_object_clones":-6,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":-5,"num_read":187,"num_read_kb":154,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":791,"num_bytes_recovered":14467215,"num_keys_recovered":32,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000604"},"pg_stats":[{"pgid":"3.1f","version":"127'440","reported_seq":4583,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865378+0000","last_change":"2024-09-17T22:59:19.038169+0000","last_active":"2024-09-17T22:59:21.865378+0000","last_peered":"2024-09-17T22:59:21.865378+0000","last_clean":"2024-09-17T22:59:21.865378+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T22:59:21.865378+0000","last_undegraded":"2024-09-17T22:59:21.865378+0000","last_fullsized":"2024-09-17T22:59:21.865378+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:01.405536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178276,"stat_sum":{"num_bytes":87780822,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":4242,"num_read_kb":23843,"num_write":442,"num_write_kb":24358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1e","version":"130'298","reported_seq":3252,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868133+0000","last_change":"2024-09-17T22:59:18.019203+0000","last_active":"2024-09-17T22:59:21.868133+0000","last_peered":"2024-09-17T22:59:21.868133+0000","last_clean":"2024-09-17T22:59:21.868133+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:59:21.868133+0000","last_undegraded":"2024-09-17T22:59:21.868133+0000","last_fullsized":"2024-09-17T22:59:21.868133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:02.822711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345557,"stat_sum":{"num_bytes":25582696,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2647,"num_read_kb":17542,"num_write":273,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1d","version":"120'534","reported_seq":9694,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867018+0000","last_change":"2024-09-17T22:59:18.017802+0000","last_active":"2024-09-17T22:59:20.867018+0000","last_peered":"2024-09-17T22:59:20.867018+0000","last_clean":"2024-09-17T22:59:20.867018+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T22:59:20.867018+0000","last_undegraded":"2024-09-17T22:59:20.867018+0000","last_fullsized":"2024-09-17T22:59:20.867018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:17:11.424158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002212122,"stat_sum":{"num_bytes":21602335,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":10247,"num_read_kb":31376,"num_write":558,"num_write_kb":6655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1c","version":"131'539","reported_seq":5651,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866792+0000","last_change":"2024-09-17T22:59:18.018154+0000","last_active":"2024-09-17T22:59:20.866792+0000","last_peered":"2024-09-17T22:59:20.866792+0000","last_clean":"2024-09-17T22:59:20.866792+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T22:59:20.866792+0000","last_undegraded":"2024-09-17T22:59:20.866792+0000","last_fullsized":"2024-09-17T22:59:20.866792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:48.792497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025587560000000001,"stat_sum":{"num_bytes":18517824,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6303,"num_read_kb":16894,"num_write":562,"num_write_kb":6173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1b","version":"135'701","reported_seq":14870,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.875255+0000","last_change":"2024-09-17T22:59:19.039165+0000","last_active":"2024-09-17T22:59:21.875255+0000","last_peered":"2024-09-17T22:59:21.875255+0000","last_clean":"2024-09-17T22:59:21.875255+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T22:59:21.875255+0000","last_undegraded":"2024-09-17T22:59:21.875255+0000","last_fullsized":"2024-09-17T22:59:21.875255+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:42.940595+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0044050690000000002,"stat_sum":{"num_bytes":19229053,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":16573,"num_read_kb":56613,"num_write":722,"num_write_kb":16994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.1a","version":"136'405","reported_seq":10182,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.956698+0000","last_change":"2024-09-17T22:59:22.865347+0000","last_active":"2024-09-17T22:59:22.956698+0000","last_peered":"2024-09-17T22:59:22.956698+0000","last_clean":"2024-09-17T22:59:22.956698+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:59:22.956698+0000","last_undegraded":"2024-09-17T22:59:22.956698+0000","last_fullsized":"2024-09-17T22:59:22.956698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:35.081547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048696600000000003,"stat_sum":{"num_bytes":21432913,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":10464,"num_read_kb":23176,"num_write":396,"num_write_kb":11995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.19","version":"127'739","reported_seq":5439,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868938+0000","last_change":"2024-09-17T22:59:18.015821+0000","last_active":"2024-09-17T22:59:21.868938+0000","last_peered":"2024-09-17T22:59:21.868938+0000","last_clean":"2024-09-17T22:59:21.868938+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:59:21.868938+0000","last_undegraded":"2024-09-17T22:59:21.868938+0000","last_fullsized":"2024-09-17T22:59:21.868938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:26.149566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066901299999999999,"stat_sum":{"num_bytes":44877072,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":6796,"num_read_kb":19698,"num_write":848,"num_write_kb":4687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.18","version":"132'476","reported_seq":5715,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868149+0000","last_change":"2024-09-17T22:59:19.041565+0000","last_active":"2024-09-17T22:59:21.868149+0000","last_peered":"2024-09-17T22:59:21.868149+0000","last_clean":"2024-09-17T22:59:21.868149+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T22:59:21.868149+0000","last_undegraded":"2024-09-17T22:59:21.868149+0000","last_fullsized":"2024-09-17T22:59:21.868149+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:25:44.508190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0054511580000000002,"stat_sum":{"num_bytes":32039559,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":4737,"num_read_kb":27532,"num_write":456,"num_write_kb":27831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.17","version":"132'674","reported_seq":4543,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865435+0000","last_change":"2024-09-17T22:59:18.025366+0000","last_active":"2024-09-17T22:59:21.865435+0000","last_peered":"2024-09-17T22:59:21.865435+0000","last_clean":"2024-09-17T22:59:21.865435+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T22:59:21.865435+0000","last_undegraded":"2024-09-17T22:59:21.865435+0000","last_fullsized":"2024-09-17T22:59:21.865435+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:21:55.359946+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055098029999999997,"stat_sum":{"num_bytes":32932558,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":5593,"num_read_kb":21131,"num_write":811,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.16","version":"127'1303","reported_seq":9540,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866783+0000","last_change":"2024-09-17T22:59:18.017250+0000","last_active":"2024-09-17T22:59:20.866783+0000","last_peered":"2024-09-17T22:59:20.866783+0000","last_clean":"2024-09-17T22:59:20.866783+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T22:59:20.866783+0000","last_undegraded":"2024-09-17T22:59:20.866783+0000","last_fullsized":"2024-09-17T22:59:20.866783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1303,"log_dups_size":0,"ondisk_log_size":1303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:39.632576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695525,"stat_sum":{"num_bytes":40767942,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":13640,"num_read_kb":44259,"num_write":1892,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.15","version":"127'433","reported_seq":4701,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868164+0000","last_change":"2024-09-17T22:59:18.024738+0000","last_active":"2024-09-17T22:59:21.868164+0000","last_peered":"2024-09-17T22:59:21.868164+0000","last_clean":"2024-09-17T22:59:21.868164+0000","last_became_active":"2024-09-17T22:58:46.132716+0000","last_became_peered":"2024-09-17T22:58:46.132716+0000","last_unstale":"2024-09-17T22:59:21.868164+0000","last_undegraded":"2024-09-17T22:59:21.868164+0000","last_fullsized":"2024-09-17T22:59:21.868164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:34.474667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061835429999999997,"stat_sum":{"num_bytes":63637947,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":4517,"num_read_kb":28303,"num_write":404,"num_write_kb":9577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.14","version":"130'462","reported_seq":2784,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.865406+0000","last_change":"2024-09-17T22:59:22.865751+0000","last_active":"2024-09-17T22:59:22.865406+0000","last_peered":"2024-09-17T22:59:22.865406+0000","last_clean":"2024-09-17T22:59:22.865406+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:59:22.865406+0000","last_undegraded":"2024-09-17T22:59:22.865406+0000","last_fullsized":"2024-09-17T22:59:22.865406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:17.502064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082622800000000003,"stat_sum":{"num_bytes":15554461,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":3496,"num_read_kb":11431,"num_write":451,"num_write_kb":5675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.915257+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T22:59:22.915257+0000","last_peered":"2024-09-17T22:59:22.915257+0000","last_clean":"2024-09-17T22:59:22.915257+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T22:59:22.915257+0000","last_undegraded":"2024-09-17T22:59:22.915257+0000","last_fullsized":"2024-09-17T22:59:22.915257+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"132'602","reported_seq":10006,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865400+0000","last_change":"2024-09-17T22:59:18.020410+0000","last_active":"2024-09-17T22:59:21.865400+0000","last_peered":"2024-09-17T22:59:21.865400+0000","last_clean":"2024-09-17T22:59:21.865400+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:59:21.865400+0000","last_undegraded":"2024-09-17T22:59:21.865400+0000","last_fullsized":"2024-09-17T22:59:21.865400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:10.984755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010349560000000001,"stat_sum":{"num_bytes":47054077,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":11985,"num_read_kb":26141,"num_write":527,"num_write_kb":10122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":248,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869786+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:59:21.869786+0000","last_peered":"2024-09-17T22:59:21.869786+0000","last_clean":"2024-09-17T22:59:21.869786+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:59:21.869786+0000","last_undegraded":"2024-09-17T22:59:21.869786+0000","last_fullsized":"2024-09-17T22:59:21.869786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"127'483","reported_seq":9581,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868645+0000","last_change":"2024-09-17T22:59:18.018460+0000","last_active":"2024-09-17T22:59:21.868645+0000","last_peered":"2024-09-17T22:59:21.868645+0000","last_clean":"2024-09-17T22:59:21.868645+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:59:21.868645+0000","last_undegraded":"2024-09-17T22:59:21.868645+0000","last_fullsized":"2024-09-17T22:59:21.868645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:19.050560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060727000000000003,"stat_sum":{"num_bytes":12802556,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":9662,"num_read_kb":29730,"num_write":502,"num_write_kb":5475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866222+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:59:20.866222+0000","last_peered":"2024-09-17T22:59:20.866222+0000","last_clean":"2024-09-17T22:59:20.866222+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:59:20.866222+0000","last_undegraded":"2024-09-17T22:59:20.866222+0000","last_fullsized":"2024-09-17T22:59:20.866222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"130'509","reported_seq":9608,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869317+0000","last_change":"2024-09-17T22:59:18.016548+0000","last_active":"2024-09-17T22:59:21.869317+0000","last_peered":"2024-09-17T22:59:21.869317+0000","last_clean":"2024-09-17T22:59:21.869317+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T22:59:21.869317+0000","last_undegraded":"2024-09-17T22:59:21.869317+0000","last_fullsized":"2024-09-17T22:59:21.869317+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:19.043740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013874099999999999,"stat_sum":{"num_bytes":10497666,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":10330,"num_read_kb":26999,"num_write":488,"num_write_kb":10537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":248,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869043+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:59:21.869043+0000","last_peered":"2024-09-17T22:59:21.869043+0000","last_clean":"2024-09-17T22:59:21.869043+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:59:21.869043+0000","last_undegraded":"2024-09-17T22:59:21.869043+0000","last_fullsized":"2024-09-17T22:59:21.869043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"130'703","reported_seq":3755,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866901+0000","last_change":"2024-09-17T22:59:18.016216+0000","last_active":"2024-09-17T22:59:20.866901+0000","last_peered":"2024-09-17T22:59:20.866901+0000","last_clean":"2024-09-17T22:59:20.866901+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T22:59:20.866901+0000","last_undegraded":"2024-09-17T22:59:20.866901+0000","last_fullsized":"2024-09-17T22:59:20.866901+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:38.503327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091405800000000004,"stat_sum":{"num_bytes":16532738,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":5442,"num_read_kb":19069,"num_write":877,"num_write_kb":12167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.3","version":"136'610","reported_seq":4989,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.950777+0000","last_change":"2024-09-17T22:59:22.865684+0000","last_active":"2024-09-17T22:59:22.950777+0000","last_peered":"2024-09-17T22:59:22.950777+0000","last_clean":"2024-09-17T22:59:22.950777+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:59:22.950777+0000","last_undegraded":"2024-09-17T22:59:22.950777+0000","last_fullsized":"2024-09-17T22:59:22.950777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:24.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072146999999999999,"stat_sum":{"num_bytes":42378885,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":5121,"num_read_kb":23740,"num_write":612,"num_write_kb":9696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"2.2","version":"21'2","reported_seq":72,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866652+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T22:59:20.866652+0000","last_peered":"2024-09-17T22:59:20.866652+0000","last_clean":"2024-09-17T22:59:20.866652+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T22:59:20.866652+0000","last_undegraded":"2024-09-17T22:59:20.866652+0000","last_fullsized":"2024-09-17T22:59:20.866652+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"130'365","reported_seq":18966,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869191+0000","last_change":"2024-09-17T22:59:18.017081+0000","last_active":"2024-09-17T22:59:21.869191+0000","last_peered":"2024-09-17T22:59:21.869191+0000","last_clean":"2024-09-17T22:59:21.869191+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T22:59:21.869191+0000","last_undegraded":"2024-09-17T22:59:21.869191+0000","last_fullsized":"2024-09-17T22:59:21.869191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:25:26.049783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001445574,"stat_sum":{"num_bytes":28389176,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":19387,"num_read_kb":25736,"num_write":332,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T22:59:23.871711+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T22:59:23.871711+0000","last_peered":"2024-09-17T22:59:23.871711+0000","last_clean":"2024-09-17T22:59:23.871711+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T22:59:23.871711+0000","last_undegraded":"2024-09-17T22:59:23.871711+0000","last_fullsized":"2024-09-17T22:59:23.871711+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'1217","reported_seq":13081,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.872814+0000","last_change":"2024-09-17T22:59:21.871845+0000","last_active":"2024-09-17T22:59:21.872814+0000","last_peered":"2024-09-17T22:59:21.872814+0000","last_clean":"2024-09-17T22:59:21.872814+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T22:59:21.872814+0000","last_undegraded":"2024-09-17T22:59:21.872814+0000","last_fullsized":"2024-09-17T22:59:21.872814+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:12:13.173957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064974900000000003,"stat_sum":{"num_bytes":17399392,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":16146,"num_read_kb":43095,"num_write":1873,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.869274+0000","last_change":"2024-09-17T22:58:46.131382+0000","last_active":"2024-09-17T22:59:21.869274+0000","last_peered":"2024-09-17T22:59:21.869274+0000","last_clean":"2024-09-17T22:59:21.869274+0000","last_became_active":"2024-09-17T22:58:46.130955+0000","last_became_peered":"2024-09-17T22:58:46.130955+0000","last_unstale":"2024-09-17T22:59:21.869274+0000","last_undegraded":"2024-09-17T22:59:21.869274+0000","last_fullsized":"2024-09-17T22:59:21.869274+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.813197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"131'818","reported_seq":6048,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.868205+0000","last_change":"2024-09-17T22:59:18.025680+0000","last_active":"2024-09-17T22:59:21.868205+0000","last_peered":"2024-09-17T22:59:21.868205+0000","last_clean":"2024-09-17T22:59:21.868205+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T22:59:21.868205+0000","last_undegraded":"2024-09-17T22:59:21.868205+0000","last_fullsized":"2024-09-17T22:59:21.868205+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:49.133668+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0076769289999999999,"stat_sum":{"num_bytes":14671190,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":6951,"num_read_kb":15969,"num_write":1052,"num_write_kb":7549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"2.3","version":"19'1","reported_seq":75,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866696+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T22:59:20.866696+0000","last_peered":"2024-09-17T22:59:20.866696+0000","last_clean":"2024-09-17T22:59:20.866696+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T22:59:20.866696+0000","last_undegraded":"2024-09-17T22:59:20.866696+0000","last_fullsized":"2024-09-17T22:59:20.866696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865206+0000","last_change":"2024-09-17T22:58:46.130584+0000","last_active":"2024-09-17T22:59:21.865206+0000","last_peered":"2024-09-17T22:59:21.865206+0000","last_clean":"2024-09-17T22:59:21.865206+0000","last_became_active":"2024-09-17T22:58:46.130202+0000","last_became_peered":"2024-09-17T22:58:46.130202+0000","last_unstale":"2024-09-17T22:59:21.865206+0000","last_undegraded":"2024-09-17T22:59:21.865206+0000","last_fullsized":"2024-09-17T22:59:21.865206+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:32.916711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"130'788","reported_seq":3752,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.874714+0000","last_change":"2024-09-17T22:59:21.872737+0000","last_active":"2024-09-17T22:59:21.874714+0000","last_peered":"2024-09-17T22:59:21.874714+0000","last_clean":"2024-09-17T22:59:21.874714+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T22:59:21.874714+0000","last_undegraded":"2024-09-17T22:59:21.874714+0000","last_fullsized":"2024-09-17T22:59:21.874714+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:00.316496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050777699999999995,"stat_sum":{"num_bytes":42088635,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":5344,"num_read_kb":13669,"num_write":1097,"num_write_kb":10146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.9","version":"136'506","reported_seq":3973,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.915627+0000","last_change":"2024-09-17T22:59:22.915627+0000","last_active":"2024-09-17T22:59:22.915627+0000","last_peered":"2024-09-17T22:59:22.915627+0000","last_clean":"2024-09-17T22:59:22.915627+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T22:59:22.915627+0000","last_undegraded":"2024-09-17T22:59:22.915627+0000","last_fullsized":"2024-09-17T22:59:22.915627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:30.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543299999999999,"stat_sum":{"num_bytes":10276669,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":5160,"num_read_kb":14284,"num_write":493,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.a","version":"136'527","reported_seq":3437,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.915830+0000","last_change":"2024-09-17T22:59:22.915830+0000","last_active":"2024-09-17T22:59:22.915830+0000","last_peered":"2024-09-17T22:59:22.915830+0000","last_clean":"2024-09-17T22:59:22.915830+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:59:22.915830+0000","last_undegraded":"2024-09-17T22:59:22.915830+0000","last_fullsized":"2024-09-17T22:59:22.915830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:19.337214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069873500000000002,"stat_sum":{"num_bytes":7753660,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":3596,"num_read_kb":19037,"num_write":620,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.b","version":"135'511","reported_seq":6167,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.875674+0000","last_change":"2024-09-17T22:59:19.034403+0000","last_active":"2024-09-17T22:59:21.875674+0000","last_peered":"2024-09-17T22:59:21.875674+0000","last_clean":"2024-09-17T22:59:21.875674+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T22:59:21.875674+0000","last_undegraded":"2024-09-17T22:59:21.875674+0000","last_fullsized":"2024-09-17T22:59:21.875674+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:27.626258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011073000000000001,"stat_sum":{"num_bytes":15510131,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":6279,"num_read_kb":16385,"num_write":514,"num_write_kb":10978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.c","version":"127'403","reported_seq":5020,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866755+0000","last_change":"2024-09-17T22:59:18.016581+0000","last_active":"2024-09-17T22:59:20.866755+0000","last_peered":"2024-09-17T22:59:20.866755+0000","last_clean":"2024-09-17T22:59:20.866755+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T22:59:20.866755+0000","last_undegraded":"2024-09-17T22:59:20.866755+0000","last_fullsized":"2024-09-17T22:59:20.866755+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:34.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652049999999999,"stat_sum":{"num_bytes":16549431,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":5438,"num_read_kb":18886,"num_write":470,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.d","version":"130'369","reported_seq":4573,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866910+0000","last_change":"2024-09-17T22:59:18.017682+0000","last_active":"2024-09-17T22:59:20.866910+0000","last_peered":"2024-09-17T22:59:20.866910+0000","last_clean":"2024-09-17T22:59:20.866910+0000","last_became_active":"2024-09-17T22:58:54.861414+0000","last_became_peered":"2024-09-17T22:58:54.861414+0000","last_unstale":"2024-09-17T22:59:20.866910+0000","last_undegraded":"2024-09-17T22:59:20.866910+0000","last_fullsized":"2024-09-17T22:59:20.866910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:14.726671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021113640000000001,"stat_sum":{"num_bytes":21422337,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5562,"num_read_kb":23049,"num_write":382,"num_write_kb":6583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.e","version":"136'327","reported_seq":1991,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.937832+0000","last_change":"2024-09-17T22:59:22.869627+0000","last_active":"2024-09-17T22:59:22.937832+0000","last_peered":"2024-09-17T22:59:22.937832+0000","last_clean":"2024-09-17T22:59:22.937832+0000","last_became_active":"2024-09-17T22:59:22.867399+0000","last_became_peered":"2024-09-17T22:59:22.867399+0000","last_unstale":"2024-09-17T22:59:22.937832+0000","last_undegraded":"2024-09-17T22:59:22.937832+0000","last_fullsized":"2024-09-17T22:59:22.937832+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:09.913651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053407599999999997,"stat_sum":{"num_bytes":12182270,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":1559,"num_read_kb":7771,"num_write":316,"num_write_kb":8441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.f","version":"136'1022","reported_seq":9008,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.865295+0000","last_change":"2024-09-17T22:59:22.865295+0000","last_active":"2024-09-17T22:59:22.865295+0000","last_peered":"2024-09-17T22:59:22.865295+0000","last_clean":"2024-09-17T22:59:22.865295+0000","last_became_active":"2024-09-17T22:58:46.132674+0000","last_became_peered":"2024-09-17T22:58:46.132674+0000","last_unstale":"2024-09-17T22:59:22.865295+0000","last_undegraded":"2024-09-17T22:59:22.865295+0000","last_fullsized":"2024-09-17T22:59:22.865295+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:10.425278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039676700000000002,"stat_sum":{"num_bytes":38643300,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":12100,"num_read_kb":34693,"num_write":1340,"num_write_kb":49269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.10","version":"127'644","reported_seq":5559,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T22:59:22.915304+0000","last_change":"2024-09-17T22:59:22.915599+0000","last_active":"2024-09-17T22:59:22.915304+0000","last_peered":"2024-09-17T22:59:22.915304+0000","last_clean":"2024-09-17T22:59:22.915304+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T22:59:22.915304+0000","last_undegraded":"2024-09-17T22:59:22.915304+0000","last_fullsized":"2024-09-17T22:59:22.915304+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:43.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045538499999999999,"stat_sum":{"num_bytes":65332590,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":5655,"num_read_kb":33513,"num_write":837,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"6"}]},{"pgid":"3.11","version":"131'490","reported_seq":4355,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.867198+0000","last_change":"2024-09-17T22:59:18.016878+0000","last_active":"2024-09-17T22:59:20.867198+0000","last_peered":"2024-09-17T22:59:20.867198+0000","last_clean":"2024-09-17T22:59:20.867198+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T22:59:20.867198+0000","last_undegraded":"2024-09-17T22:59:20.867198+0000","last_fullsized":"2024-09-17T22:59:20.867198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:07.569383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647997,"stat_sum":{"num_bytes":13042207,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":4434,"num_read_kb":17010,"num_write":467,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.12","version":"120'753","reported_seq":13221,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T22:59:21.865204+0000","last_change":"2024-09-17T22:59:18.021520+0000","last_active":"2024-09-17T22:59:21.865204+0000","last_peered":"2024-09-17T22:59:21.865204+0000","last_clean":"2024-09-17T22:59:21.865204+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:59:21.865204+0000","last_undegraded":"2024-09-17T22:59:21.865204+0000","last_fullsized":"2024-09-17T22:59:21.865204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:31.309813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025369390000000002,"stat_sum":{"num_bytes":26054365,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":15096,"num_read_kb":33022,"num_write":952,"num_write_kb":9478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"5"}]},{"pgid":"3.13","version":"130'607","reported_seq":8883,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-17T22:59:20.866147+0000","last_change":"2024-09-17T22:59:18.012873+0000","last_active":"2024-09-17T22:59:20.866147+0000","last_peered":"2024-09-17T22:59:20.866147+0000","last_clean":"2024-09-17T22:59:20.866147+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:59:20.866147+0000","last_undegraded":"2024-09-17T22:59:20.866147+0000","last_fullsized":"2024-09-17T22:59:20.866147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:19.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074441799999999995,"stat_sum":{"num_bytes":35205111,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":10373,"num_read_kb":26648,"num_write":676,"num_write_kb":10339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":917741568,"num_objects":1170,"num_object_clones":226,"num_object_copies":2340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1170,"num_whiteouts":620,"num_read":254875,"num_read_kb":790244,"num_write":22242,"num_write_kb":366003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143126528,"data_stored":157088863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":19258,"ondisk_log_size":19258,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50676,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321164,"statfs":{"total":96636764160,"available":96584871936,"internally_reserved":0,"allocated":11427840,"data_stored":11364722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678032,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63612,"kb_used_data":22308,"kb_used_omap":17,"kb_used_meta":41262,"kb_avail":94308228,"statfs":{"total":96636764160,"available":96571625472,"internally_reserved":0,"allocated":22843392,"data_stored":23898875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":42252486},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76624,"kb_used_data":31544,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94295216,"statfs":{"total":96636764160,"available":96558301184,"internally_reserved":0,"allocated":32301056,"data_stored":35454518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75536,"kb_used_data":17976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94296304,"statfs":{"total":96636764160,"available":96559415296,"internally_reserved":0,"allocated":18407424,"data_stored":18579293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977412,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":5572,"kb_used_omap":14,"kb_used_meta":28401,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":5705728,"data_stored":5277893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":29083050},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":13372,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":13692928,"data_stored":13641694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83868,"kb_used_data":26312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287972,"statfs":{"total":96636764160,"available":96550883328,"internally_reserved":0,"allocated":26943488,"data_stored":27891631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84532,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94287308,"statfs":{"total":96636764160,"available":96550203392,"internally_reserved":0,"allocated":27623424,"data_stored":30121396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24748032,"data_stored":26528606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12091392,"data_stored":12869282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3203072,"data_stored":3630044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16805888,"data_stored":17806636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30097408,"data_stored":34092212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21241856,"data_stored":23126463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9506816,"data_stored":10276985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25432064,"data_stored":28758635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:59:26.802 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T22:59:26.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:59:26.802 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T22:59:26.802 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:59:26.851 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:26.848+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:26.892 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:59:26.915 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:26.916 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:26.916 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:26.916 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:26.916 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:26.925 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:26.952 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:26.949+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:26.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:27.052 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:27.049+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:27.071 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.316236816, 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.072 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.073 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:27.075 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:27.076 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.857066803999999, 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:27.077 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.078 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.079 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.080 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.081 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.851957720000001, 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.082 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.083 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:27.084 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.085 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.086 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.708037818000001, 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.087 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.088 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.089 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [3813569~36671,4030464~65536,4530705~42381,4871284~53716,12790648~38020,15826239~34415], data_size: 270739, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15860654, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.704241586000002, 2024-09-17T22:59:27.091 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021559650000000001, 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364339+0000", 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022050699999999999 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.093 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364352+0000", 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3560000000000001e-05 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364363+0000", 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0764000000000001e-05 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364449+0000", 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6137000000000006e-05 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.094 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364464+0000", 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4712000000000001e-05 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.366274+0000", 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018097359999999999 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.585320461000002, 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.096 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.097 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:27.098 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [1343488~49152,1425408~114688,1572864~114688,1720320~114688,1867776~114688,2015232~114688,2162688~114688,2310144~49152], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 32.581890561999998, 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002200498, 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.099 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.100 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487192+0000", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072275300000000002 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487207+0000", 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4586000000000001e-05 2024-09-17T22:59:27.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487218+0000", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1314999999999999e-05 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487316+0000", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8603000000000004e-05 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487332+0000", 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5532e-05 2024-09-17T22:59:27.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.488669+0000", 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013373230000000001 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 29.195479452000001, 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.103 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.104 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:27.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:27.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:27.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:27.108 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.156866439000002, 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.110 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:27.111 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:27.112 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.113 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.114 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.155100511000001, 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:27.116 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.117 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.118 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.119 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.120 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.121 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.122 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.149664213000001, 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.123 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.124 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:27.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.126 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.127 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.124963728000001, 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.128 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:27.129 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:27.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.131 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 20.12076467, 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.132 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.133 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:27.134 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.135 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.136 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.137 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.125025634, 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.138 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.139 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:27.141 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:199116 3.17 e121/108 3:ed33c955:::rbd_header.1188b9dbd70:head v 121'628, mlcod=120'626)", 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.12437199, 2024-09-17T22:59:27.142 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00221332, 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.143 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943988+0000", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943995+0000", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1239999999999999e-06 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944093+0000", 2024-09-17T22:59:27.144 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7671999999999996e-05 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944104+0000", 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944212+0000", 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108087 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.145 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944237+0000", 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4841000000000002e-05 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946157+0000", 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019206939999999999 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946201+0000", 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3225e-05 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.146 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 19.001662459999999, 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.147 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.148 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.150 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200096 3.11 3:8e35b8be:::rbd_data.11884120dce1.000000000000000f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.9829483070000009, 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022879160000000001, 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.151 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200096 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.152 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085412+0000", 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0559999999999999e-06 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085435+0000", 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3000999999999999e-05 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.153 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085451+0000", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5899000000000001e-05 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085459+0000", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3990000000000004e-06 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085566+0000", 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-17T22:59:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085717+0000", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015122599999999999 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085826+0000", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010882699999999999 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.086549+0000", 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072324900000000003 2024-09-17T22:59:27.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087552+0000", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010025909999999999 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087652+0000", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010069300000000001 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087699+0000", 2024-09-17T22:59:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6470000000000001e-05 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200163 3.11 3:8ca7bd04:::rbd_header.1188b5c0f048:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.9357128479999997, 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022726589999999998, 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200163 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132649+0000", 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132654+0000", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132657+0000", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.106e-06 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132891+0000", 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000234212 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132953+0000", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2217000000000004e-05 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133128+0000", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017510200000000001 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133539+0000", 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041042999999999999 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134734+0000", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011958190000000001 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134863+0000", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128555 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134919+0000", 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6168999999999997e-05 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200744 3.11 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.1246547730000001, 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002277519, 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200744 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:59:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943716+0000", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0552e-05 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943729+0000", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2553e-05 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943736+0000", 2024-09-17T22:59:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3350000000000001e-06 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943784+0000", 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7678e-05 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943862+0000", 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288999999999995e-05 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943933+0000", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0761999999999998e-05 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944996+0000", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001062849 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945883+0000", 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088707900000000004 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945939+0000", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6242999999999997e-05 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945982+0000", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200745 3.11 3:8e06f6ab:::rbd_data.1188d23b4ac7.0000000000000013:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 2.1246052230000001, 2024-09-17T22:59:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002340698, 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200745 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943755+0000", 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943758+0000", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0280000000000001e-06 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943765+0000", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0929999999999997e-06 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943771+0000", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9179999999999999e-06 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:27.171 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944323+0000", 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055252999999999997 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944465+0000", 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014123799999999999 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944593+0000", 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012883099999999999 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945529+0000", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093524199999999997 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946020+0000", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00049166900000000003 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946051+0000", 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0681e-05 2024-09-17T22:59:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946095+0000", 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3897999999999999e-05 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:27.174 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:27.175 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:27.175 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:27.176 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:27.176 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:27.177 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:27.177 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:27.177 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:27.178 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:27.178 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:27.178 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "epoch": 140, 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "fsid": "33081d94-d15e-4bd4-ba4c-30022eb1be7a", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "created": "2024-09-17T22:56:50.056303+0000", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "modified": "2024-09-17T22:59:26.866422+0000", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "last_up_change": "2024-09-17T22:59:09.941537+0000", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "last_in_change": "2024-09-17T22:58:51.164342+0000", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_num": 5804032, 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_set": [ 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "nodeep-scrub", 2024-09-17T22:59:27.179 INFO:teuthology.orchestra.run.smithi159.stdout: "pglog_hardlimit", 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "purged_snapdirs", 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_deletes", 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "sortbitwise" 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_version": 7, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_snapshot": "", 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_max": 3, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd": 8, 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:59:27.180 INFO:teuthology.orchestra.run.smithi159.stdout: "min_compat_client": "luminous", 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "require_osd_release": "squid", 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_crimson": false, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "pools": [ 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 1, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": ".mgr", 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:02.221581+0000", 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool", 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:27.181 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 1, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 1, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 1, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 1, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 1, 2024-09-17T22:59:27.182 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "17", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:27.183 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:27.184 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:27.185 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "options": { 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_max": 32, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_min": 1 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": {} 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:27.186 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 8, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 8, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.25, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 2, 2024-09-17T22:59:27.187 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "rbd", 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:08.859324+0000", 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:27.188 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 8, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 8, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 8, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 8, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 8, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:27.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "21", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 2, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 21, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:27.190 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:27.192 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 2.9900000095367432, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 2.9900000095367432, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2.630000114440918, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2.630000114440918, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:27.193 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "pool_client.0", 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:17.992936+0000", 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:27.194 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 32, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 32, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 32, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 32, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 32, 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "140", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 54, 2024-09-17T22:59:27.196 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 140, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:27.197 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:27.198 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.75, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.75, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 1, 2024-09-17T22:59:27.199 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.75, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.75, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "892321c3-409f-40e6-bfe1-c87f464d030f", 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6808", 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6809", 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6810", 2024-09-17T22:59:27.202 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6811", 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6814", 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6815", 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6812", 2024-09-17T22:59:27.204 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6813", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6809/2087020665", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6811/2087020665", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6815/2087020665", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6813/2087020665", 2024-09-17T22:59:27.205 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "cd777f15-a6ad-4f0c-820b-ef529dd9f008", 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.206 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6800", 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.207 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6801", 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6802", 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6803", 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6806", 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6807", 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6804", 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6805", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6801/237398697", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6803/237398697", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6807/237398697", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6805/237398697", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.211 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "bd68e905-6e66-4735-9de8-66a99530420f", 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 14, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 74, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 123, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 130, 2024-09-17T22:59:27.212 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 75, 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6816", 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6817", 2024-09-17T22:59:27.213 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6818", 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6819", 2024-09-17T22:59:27.214 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6822", 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6823", 2024-09-17T22:59:27.215 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6820", 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.216 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6821", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6817/3362624831", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6819/3362624831", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6823/3362624831", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6821/3362624831", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "28b3b7ca-d7d7-4ed9-8dfe-113d3579209c", 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6824", 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6825", 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.219 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6826", 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6827", 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.220 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6830", 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6831", 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.221 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6828", 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6829", 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6825/2987856795", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6827/2987856795", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6831/2987856795", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6829/2987856795", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "d379c252-e66a-4d0e-af0a-81f6bda48c08", 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.223 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 135, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6826", 2024-09-17T22:59:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6827", 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6828", 2024-09-17T22:59:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6829", 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6832", 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6833", 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6830", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6831", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6827/2123531187", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6829/2123531187", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6833/2123531187", 2024-09-17T22:59:27.228 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6831/2123531187", 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "e78a100f-1ff3-4af6-abd8-a069030ec951", 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 15, 2024-09-17T22:59:27.229 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 45, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 71, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 126, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 46, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6818", 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6819", 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6820", 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.231 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6821", 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6824", 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6825", 2024-09-17T22:59:27.232 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6822", 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.233 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6823", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6819/587621150", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6821/587621150", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6825/587621150", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6823/587621150", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "fc1af848-dfaa-4e40-a0af-8982c70f7ea9", 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.235 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6810", 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6811", 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.236 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6812", 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6813", 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.237 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6816", 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6817", 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.238 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6814", 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6815", 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.239 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6811/1505991260", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6813/1505991260", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6817/1505991260", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6815/1505991260", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "ccd75a18-ded4-4f75-a09a-926da5362a7f", 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:27.240 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 0.1603546142578125, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 13, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 75, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6800", 2024-09-17T22:59:27.241 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6801", 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6802", 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6803", 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6806", 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6807", 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6804", 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6805", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6801/585405482", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6803/585405482", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6807/585405482", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6805/585405482", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_xinfo": [ 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.101775+0000", 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:59:27.246 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.929708+0000", 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.562720+0000", 2024-09-17T22:59:27.247 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 75 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.569375+0000", 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:59:27.248 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:57:00.002784+0000", 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.249 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.775667+0000", 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 46 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.558418+0000", 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.840790+0000", 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap": [ 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "pgid": "3.9", 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:27.251 INFO:teuthology.orchestra.run.smithi159.stdout: 6, 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: 3 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: "pgid": "3.11", 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: 5, 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: 2 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_items": [], 2024-09-17T22:59:27.252 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_primaries": [], 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_temp": [], 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_temp": [], 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "blocklist": {}, 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "range_blocklist": {}, 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profiles": { 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "default": { 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-failure-domain": "osd", 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "k": "2", 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "m": "1", 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin": "jerasure", 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "technique": "reed_sol_van" 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:27.253 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps_queue": [ 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 24, 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 10 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "new_removed_snaps": [ 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:59:27.254 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 24, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 1 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "new_purged_snaps": [], 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_node_flags": {}, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "device_class_flags": {}, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode": { 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_enabled": false, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_bucket_count": 0, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:59:27.255 INFO:teuthology.orchestra.run.smithi159.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:59:27.256 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_bucket": 0 2024-09-17T22:59:27.256 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:27.256 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:27.256 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:27.258 INFO:tasks.thrashosds.thrasher:j is {'epoch': 140, 'fsid': '33081d94-d15e-4bd4-ba4c-30022eb1be7a', 'created': '2024-09-17T22:56:50.056303+0000', 'modified': '2024-09-17T22:59:26.866422+0000', 'last_up_change': '2024-09-17T22:59:09.941537+0000', 'last_in_change': '2024-09-17T22:58:51.164342+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:57:02.221581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:57:08.859324+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:57:17.992936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 140, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': '892321c3-409f-40e6-bfe1-c87f464d030f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6808', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6809', 'nonce': 2087020665}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6810', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6811', 'nonce': 2087020665}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6814', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6815', 'nonce': 2087020665}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6812', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6813', 'nonce': 2087020665}]}, 'public_addr': '172.21.15.159:6809/2087020665', 'cluster_addr': '172.21.15.159:6811/2087020665', 'heartbeat_back_addr': '172.21.15.159:6815/2087020665', 'heartbeat_front_addr': '172.21.15.159:6813/2087020665', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'cd777f15-a6ad-4f0c-820b-ef529dd9f008', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6800', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6801', 'nonce': 237398697}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6802', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6803', 'nonce': 237398697}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6806', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6807', 'nonce': 237398697}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6804', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6805', 'nonce': 237398697}]}, 'public_addr': '172.21.15.159:6801/237398697', 'cluster_addr': '172.21.15.159:6803/237398697', 'heartbeat_back_addr': '172.21.15.159:6807/237398697', 'heartbeat_front_addr': '172.21.15.159:6805/237398697', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bd68e905-6e66-4735-9de8-66a99530420f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 74, 'up_from': 123, 'up_thru': 130, 'down_at': 75, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6816', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6817', 'nonce': 3362624831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6818', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6819', 'nonce': 3362624831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6822', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6823', 'nonce': 3362624831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6820', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6821', 'nonce': 3362624831}]}, 'public_addr': '172.21.15.159:6817/3362624831', 'cluster_addr': '172.21.15.159:6819/3362624831', 'heartbeat_back_addr': '172.21.15.159:6823/3362624831', 'heartbeat_front_addr': '172.21.15.159:6821/3362624831', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '28b3b7ca-d7d7-4ed9-8dfe-113d3579209c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6824', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6825', 'nonce': 2987856795}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6826', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6827', 'nonce': 2987856795}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6830', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6831', 'nonce': 2987856795}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6828', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6829', 'nonce': 2987856795}]}, 'public_addr': '172.21.15.159:6825/2987856795', 'cluster_addr': '172.21.15.159:6827/2987856795', 'heartbeat_back_addr': '172.21.15.159:6831/2987856795', 'heartbeat_front_addr': '172.21.15.159:6829/2987856795', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd379c252-e66a-4d0e-af0a-81f6bda48c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2123531187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2123531187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2123531187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2123531187}]}, 'public_addr': '172.21.15.171:6827/2123531187', 'cluster_addr': '172.21.15.171:6829/2123531187', 'heartbeat_back_addr': '172.21.15.171:6833/2123531187', 'heartbeat_front_addr': '172.21.15.171:6831/2123531187', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e78a100f-1ff3-4af6-abd8-a069030ec951', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 45, 'up_from': 71, 'up_thru': 126, 'down_at': 46, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 587621150}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 587621150}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 587621150}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 587621150}]}, 'public_addr': '172.21.15.171:6819/587621150', 'cluster_addr': '172.21.15.171:6821/587621150', 'heartbeat_back_addr': '172.21.15.171:6825/587621150', 'heartbeat_front_addr': '172.21.15.171:6823/587621150', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'fc1af848-dfaa-4e40-a0af-8982c70f7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 1505991260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 1505991260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1505991260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 1505991260}]}, 'public_addr': '172.21.15.171:6811/1505991260', 'cluster_addr': '172.21.15.171:6813/1505991260', 'heartbeat_back_addr': '172.21.15.171:6817/1505991260', 'heartbeat_front_addr': '172.21.15.171:6815/1505991260', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ccd75a18-ded4-4f75-a09a-926da5362a7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1603546142578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 585405482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 585405482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 585405482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 585405482}]}, 'public_addr': '172.21.15.171:6801/585405482', 'cluster_addr': '172.21.15.171:6803/585405482', 'heartbeat_back_addr': '172.21.15.171:6807/585405482', 'heartbeat_front_addr': '172.21.15.171:6805/585405482', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.101775+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.929708+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.562720+0000', 'dead_epoch': 75}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.569375+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:57:00.002784+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.775667+0000', 'dead_epoch': 46}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.558418+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.840790+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.9', 'osds': [6, 3]}, {'pgid': '3.11', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 10}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:59:27.258 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T22:59:27.258 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:27.173+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:27.259 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:27.259 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:27.259 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:27.260 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:27.261 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:27.276 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:27.273+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:27.377 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:27.374+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:27.400 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:27.401 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:27.409 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:27.494 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:27.492+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:27.554 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:27.554 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:27.562 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:27.603 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:27.601+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:27.679 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:27.677+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:27.706 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:27.706 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:27.714 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:27.799 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:27.796+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:27.858 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:27.858 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:27.866 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:27.878 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:59:27.881 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:27.878+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:27.982 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:27.979+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:27.991 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:27.991 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.000 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:28.083 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:28.081+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:28.104 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.104 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.112 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:28.184 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:28.181+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:28.255 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.255 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.264 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:28.285 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:28.283+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:28.367 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.367 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.375 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:28.420 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:28.418+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:28.479 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.479 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.486 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:28.483+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:28.487 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:28.587 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:28.584+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:28.631 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.631 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.640 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:28.687 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:28.685+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:28.742 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.742 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.751 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:28.789 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:28.786+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:28.889 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:28.887+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:28.901 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:28.901 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:28.910 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:28.990 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:28.987+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:29.024 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:29.024 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:29.032 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:29.117 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:29.114+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:29.177 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:29.177 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:29.186 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:29.191 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:29.189+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:29.262 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:29.292 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:29.289+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:29.330 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:29.330 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:29.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:29.383 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:29.384 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:29.400 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:29.416 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:29.416 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:29.423 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:29.420+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:29.432 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:29.432 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:29.448 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:29.465 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:29.465 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:29.481 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:29.481 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:29.484 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:29.484 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:29.493 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:29.494 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:29.492+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:29.497 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:29.498 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:29.596 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:29.593+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:29.637 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:29.638 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:29.638 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:29.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:29.638 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:29.646 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:29.731 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:29.728+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534563338, 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.791 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.792 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:29.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:29.794 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.795 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.53454733300001, 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:29.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.797 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.798 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.799 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534514563, 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.800 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.801 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.802 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:29.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:29.804 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534486825, 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.805 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.806 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.807 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:29.808 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534464406, 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:29.810 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:29.811 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.812 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:29.813 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.814 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:29.815 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534442126, 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:29.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.817 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.818 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.819 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.534234721, 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.820 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.821 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:29.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.823 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:29.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.533688106, 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:29.825 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:29.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.53261691, 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.830 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:29.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.834 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.532605455, 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.835 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:29.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.53259447799999, 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.840 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:29.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.530520346, 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.530508032, 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.849 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:29.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:29.853 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.530494795, 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:29.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:29.857 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.53048396600001, 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.858 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:29.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.52852509100001, 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:29.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:29.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:29.867 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.528500733, 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:29.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:29.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.872 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.52847151500001, 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:29.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:29.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:29.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 113.52769184500001, 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.877 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:29.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:29.881 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.908833306999998, 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:29.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:29.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:29.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:29.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:29.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:29.889 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:29.891 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:29.891 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:29.888+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:29.898 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:59:29.991 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:29.988+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:30.039 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:30.039 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:30.039 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:30.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:30.131 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:30.128+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:30.191 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.191 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:30.191 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:30.192 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:30.192 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:30.193 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:30.190+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:30.200 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:30.294 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:30.291+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.346 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 146.48221197300001, 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:30.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:30.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:30.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:30.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.354 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.255372846, 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.355 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.356 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:30.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:30.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:30.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:30.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:30.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.25535432, 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.364 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:30.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.255336148, 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.369 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.370 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:30.371 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:30.372 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.373 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.25531832, 2024-09-17T22:59:30.374 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:30.376 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.377 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.378 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.379 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 119.255003187, 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.380 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.381 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.382 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.383 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.384 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773178852999997, 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.385 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:30.386 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.387 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.388 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.389 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:30.390 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773169224999997, 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.391 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.392 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.393 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:30.394 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.395 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773159876999998, 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:30.396 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:30.397 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.398 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.399 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773149955000001, 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:30.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:30.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.406 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:30.407 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773136499000003, 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:30.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773126687000001, 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.413 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:30.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:30.418 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773096318, 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.419 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.420 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:30.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:30.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:30.424 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773083995, 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:30.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.429 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773074487999999, 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.430 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:30.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:30.435 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773064447000003, 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:30.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:30.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:30.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773055163000002, 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.441 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:30.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:30.446 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773045668999998, 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.447 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:30.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:30.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773034400999997, 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:30.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.457 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.773024995999997, 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:30.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.463 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:30.465 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:30.465 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:30.461+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:30.565 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:30.562+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:30.610 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:30.610 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:30.611 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:30.619 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:30.703 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:30.700+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:30.762 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:30.763 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:30.763 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:30.768 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:30.765+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:30.771 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:30.869 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:30.866+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 37.170610154999999, 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:30.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:30.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.711440142999997, 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.931 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:30.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.706331059, 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:30.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:30.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.562411157, 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:30.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [3813569~36671,4030464~65536,4530705~42381,4871284~53716,12790648~38020,15826239~34415], data_size: 270739, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15860654, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.558614925000001, 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021559650000000001, 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364118+0000", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364339+0000", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00022050699999999999 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364352+0000", 2024-09-17T22:59:30.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3560000000000001e-05 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364363+0000", 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0764000000000001e-05 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364449+0000", 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6137000000000006e-05 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.364464+0000", 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4712000000000001e-05 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.366274+0000", 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018097359999999999 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.439693800000001, 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:30.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [1343488~49152,1425408~114688,1572864~114688,1720320~114688,1867776~114688,2015232~114688,2162688~114688,2310144~49152], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.436263900999997, 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002200498, 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.486469+0000", 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487192+0000", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072275300000000002 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487207+0000", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4586000000000001e-05 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487218+0000", 2024-09-17T22:59:30.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1314999999999999e-05 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487316+0000", 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8603000000000004e-05 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.487332+0000", 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5532e-05 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.488669+0000", 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0013373230000000001 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 33.049852790999999, 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:30.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:30.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:30.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 28.011239778, 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:30.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:30.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:30.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.009473849999999, 2024-09-17T22:59:30.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:30.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.004037552, 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:30.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.979337066999999, 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.986 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:30.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:30.990 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.975138008999998, 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:30.991 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:30.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:30.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:30.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.979398972999999, 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:30.997 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:30.998 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:30.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:31.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:199116 3.17 e121/108 3:ed33c955:::rbd_header.1188b9dbd70:head v 121'628, mlcod=120'626)", 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.978745328999999, 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00221332, 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943988+0000", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943995+0000", 2024-09-17T22:59:31.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1239999999999999e-06 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944093+0000", 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7671999999999996e-05 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944104+0000", 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944212+0000", 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108087 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944237+0000", 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4841000000000002e-05 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946157+0000", 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019206939999999999 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946201+0000", 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3225e-05 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 22.856035799000001, 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.006 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.007 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:31.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:31.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.010 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200096 3.11 3:8e35b8be:::rbd_data.11884120dce1.000000000000000f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.837321645999999, 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022879160000000001, 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:31.011 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200096 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:31.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085412+0000", 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0559999999999999e-06 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085435+0000", 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3000999999999999e-05 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085451+0000", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5899000000000001e-05 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085459+0000", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3990000000000004e-06 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085566+0000", 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085717+0000", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015122599999999999 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085826+0000", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010882699999999999 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.086549+0000", 2024-09-17T22:59:31.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072324900000000003 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087552+0000", 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010025909999999999 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087652+0000", 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010069300000000001 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087699+0000", 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6470000000000001e-05 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200163 3.11 3:8ca7bd04:::rbd_header.1188b5c0f048:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.790086187, 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022726589999999998, 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200163 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132649+0000", 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132654+0000", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132657+0000", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.106e-06 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132891+0000", 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000234212 2024-09-17T22:59:31.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132953+0000", 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2217000000000004e-05 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133128+0000", 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017510200000000001 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:31.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133539+0000", 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041042999999999999 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134734+0000", 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011958190000000001 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134863+0000", 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128555 2024-09-17T22:59:31.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134919+0000", 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6168999999999997e-05 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200744 3.11 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.979028112, 2024-09-17T22:59:31.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002277519, 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200744 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.024 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.025 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943716+0000", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0552e-05 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943729+0000", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2553e-05 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943736+0000", 2024-09-17T22:59:31.026 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3350000000000001e-06 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943784+0000", 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7678e-05 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943862+0000", 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288999999999995e-05 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:31.027 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943933+0000", 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0761999999999998e-05 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944996+0000", 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001062849 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945883+0000", 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088707900000000004 2024-09-17T22:59:31.028 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945939+0000", 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6242999999999997e-05 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945982+0000", 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.029 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200745 3.11 3:8e06f6ab:::rbd_data.1188d23b4ac7.0000000000000013:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 5.978978562, 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002340698, 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200745 2024-09-17T22:59:31.030 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.031 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943755+0000", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943758+0000", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0280000000000001e-06 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943765+0000", 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0929999999999997e-06 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.032 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943771+0000", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9179999999999999e-06 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944323+0000", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055252999999999997 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944465+0000", 2024-09-17T22:59:31.033 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014123799999999999 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944593+0000", 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012883099999999999 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945529+0000", 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093524199999999997 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.034 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946020+0000", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00049166900000000003 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946051+0000", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0681e-05 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946095+0000", 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3897999999999999e-05 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.035 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:31.036 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:31.036 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:31.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:31.038 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:31.034+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:31.138 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:31.136+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:31.142 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:31.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:31.142 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:31.142 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:31.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:31.239 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:31.237+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:31.294 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:31.295 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:31.295 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:31.295 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:31.295 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:31.303 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:31.347 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:31.345+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26, version: 76'382, data_included: [0~631961,1179648~65536,4194304~131072], data_size: 828569, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26@76'382, size: 7340032, copy_subset: [(0, 7340032)], clone_subset: {}, snapset: 26=[]:{26: [26]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7340032, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.449638276000002, 2024-09-17T22:59:31.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021080690000000002, 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:31.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954950+0000", 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955691+0000", 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074070899999999999 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955710+0000", 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9463999999999999e-05 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955731+0000", 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0437000000000001e-05 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955812+0000", 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1221e-05 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955822+0000", 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0016e-05 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.957057+0000", 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012356509999999999 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.412 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.420057005, 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001379094, 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3799999999999997e-07 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985075+0000", 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054357699999999995 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985092+0000", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7041000000000002e-05 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985103+0000", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0740999999999999e-05 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985162+0000", 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9290999999999999e-05 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985170+0000", 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7579999999999992e-06 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985910+0000", 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074014800000000004 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.417715252000001, 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013103189999999999, 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986873+0000", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987326+0000", 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00045334100000000001 2024-09-17T22:59:31.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987343+0000", 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6495e-05 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987354+0000", 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0791999999999999e-05 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987411+0000", 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7445000000000003e-05 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987419+0000", 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0139999999999998e-06 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.988183+0000", 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00076374600000000004 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.420 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35, version: 91'439, data_included: [12288~28672,102400~28672,143360~28672,233472~28672,274432~28672,364544~28672,405504~28672,495616~28672,536576~28672,626688~28672,667648~28672,757760~28672,798720~28672,888832~28672,929792~20480,1019904~28672], data_size: 450560, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35@91'439, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 35=[]:{35: [35]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.400949473000001, 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017163370000000001, 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003639+0000", 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.71e-07 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003943+0000", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00030434599999999999 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003969+0000", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5700000000000001e-05 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003994+0000", 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4416000000000001e-05 2024-09-17T22:59:31.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004076+0000", 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2137000000000003e-05 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004086+0000", 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0271999999999999e-05 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.005355+0000", 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001268595 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1528 3.4 e124/123 3:2363738a:::rbd_data.1188b9dbd70.0000000000000032:23 v 124'482, mlcod=115'479)", 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.35045985, 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022419990000000002, 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.425 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054135+0000", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2750000000000001e-06 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054146+0000", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0829000000000001e-05 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054153+0000", 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0099999999999998e-06 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054191+0000", 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8550999999999997e-05 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054205+0000", 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3891e-05 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056337+0000", 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021323190000000001 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056370+0000", 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2455e-05 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1529 3.4 e124/123 3:220bdc83:::rbd_data.1188f4e2c696.000000000000010a:24 v 124'484, mlcod=115'480)", 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.347381616, 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025955399999999999, 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057207+0000", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7899999999999998e-07 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057212+0000", 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0479999999999998e-06 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057221+0000", 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057227+0000", 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3369999999999996e-06 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057277+0000", 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9292000000000002e-05 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057290+0000", 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059759+0000", 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0024690379999999998 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059802+0000", 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2106e-05 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1530 3.4 e124/123 3:222138c9:::rbd_data.1188f4e2c696.000000000000003e:24 v 124'486, mlcod=115'480)", 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.346892144000002, 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023558860000000002, 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.434 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057703+0000", 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8859999999999998e-06 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057711+0000", 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2949999999999994e-06 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057717+0000", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4779999999999999e-06 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057755+0000", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8606000000000003e-05 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057767+0000", 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1484999999999999e-05 2024-09-17T22:59:31.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060021+0000", 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022539080000000002 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060051+0000", 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0679000000000002e-05 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1531 3.4 e124/123 3:2668e2f4:::rbd_data.1188f4e2c696.0000000000000042:24 v 124'488, mlcod=124'484)", 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.344149964, 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001073601, 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060444+0000", 2024-09-17T22:59:31.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.237e-06 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060454+0000", 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0824e-05 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060461+0000", 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060501+0000", 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0577000000000001e-05 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060515+0000", 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3429e-05 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061462+0000", 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00094694600000000005 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061511+0000", 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9435999999999999e-05 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1534 3.4 e124/123 3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30 v 124'494, mlcod=124'490)", 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.334226945000001, 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001260253, 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.443 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.444 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.68e-07 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070367+0000", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8780000000000003e-06 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070377+0000", 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7440000000000002e-06 2024-09-17T22:59:31.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070384+0000", 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9140000000000002e-06 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070425+0000", 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0800999999999998e-05 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.446 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070446+0000", 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1603e-05 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071578+0000", 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001131878 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071621+0000", 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2766999999999999e-05 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.447 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1539 3.4 e124/123 3:263d8630:::rbd_data.11881ee9ab63.000000000000006a:36 v 124'504, mlcod=124'498)", 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 20.325141852000002, 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001105275, 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.448 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079447+0000", 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.449 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079453+0000", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079461+0000", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1699999999999997e-06 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079468+0000", 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5570000000000001e-06 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.450 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079506+0000", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7540999999999999e-05 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079530+0000", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4508999999999999e-05 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080522+0000", 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099224300000000007 2024-09-17T22:59:31.451 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080551+0000", 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8736999999999999e-05 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.11 125/124 [PushOp(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head, version: 48'115, data_included: [0~262144,524288~287322,819200~49152,942080~49152], data_size: 647770, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head@48'115, size: 1356976, copy_subset: [(0, 1356976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1356976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:31.452 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 19.269750962, 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002060559, 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:31.453 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135765+0000", 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00092785999999999997 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.454 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135788+0000", 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2623e-05 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135808+0000", 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-05 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135897+0000", 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8865999999999999e-05 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.455 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135909+0000", 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2482e-05 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.136897+0000", 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098805099999999999 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200097 3.4 e130/123 3:26073371:::rbd_data.11884120dce1.0000000000000013:head v 130'509, mlcod=124'506)", 2024-09-17T22:59:31.456 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.318723793, 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011580830000000001, 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.457 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3599999999999999e-07 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085870+0000", 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3940000000000004e-06 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.458 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085888+0000", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8505999999999999e-05 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085897+0000", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3549999999999994e-06 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085978+0000", 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0677e-05 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.459 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085999+0000", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1192e-05 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.086969+0000", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00096941300000000002 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.087022+0000", 2024-09-17T22:59:31.460 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.321e-05 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200163 3.11 e130/126 3:8ca7bd04:::rbd_header.1188b5c0f048:head v 130'473, mlcod=130'471)", 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.271365214999999, 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011031330000000001, 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133223+0000", 2024-09-17T22:59:31.462 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1500000000000004e-07 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133230+0000", 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.477e-06 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133240+0000", 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.463 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133246+0000", 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9839999999999998e-06 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133292+0000", 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6783999999999998e-05 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133311+0000", 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8919e-05 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.464 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.134295+0000", 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000983252 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.134326+0000", 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1061e-05 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPushReply(3.1f 130/129 [PushReplyOp(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f)])", 2024-09-17T22:59:31.465 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.151954315999999, 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011045429999999999, 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.466 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252634+0000", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7400000000000002e-07 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252639+0000", 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9760000000000003e-06 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.467 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252652+0000", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2744e-05 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252663+0000", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0869999999999999e-05 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252716+0000", 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2947e-05 2024-09-17T22:59:31.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.253738+0000", 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0010222320000000001 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200744 3.11 e138/126 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head v 138'496, mlcod=136'494)", 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 6.4601002730000001, 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001254745, 2024-09-17T22:59:31.469 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944488+0000", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7799999999999997e-07 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944494+0000", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0290000000000002e-06 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944510+0000", 2024-09-17T22:59:31.471 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6548000000000001e-05 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944520+0000", 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2830000000000007e-06 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944611+0000", 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1547999999999998e-05 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.472 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944633+0000", 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1599000000000001e-05 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.945699+0000", 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001066333 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.945742+0000", 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3127000000000002e-05 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.473 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1611 3.0 e139/126 3:0562d798:::rbd_data.1188457bc75a.000000000000000d:19 v 139'368, mlcod=130'365)", 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5271500949999997, 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0014449560000000001, 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:31.474 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-07 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.475 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877444+0000", 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1809999999999998e-06 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877488+0000", 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3646000000000002e-05 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877497+0000", 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9919999999999996e-06 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.476 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877600+0000", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010299800000000001 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877624+0000", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4329e-05 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878837+0000", 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001212631 2024-09-17T22:59:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878883+0000", 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5911000000000001e-05 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1612 3.4 e139/123 3:268a8c58:::rbd_data.1188457bc75a.0000000000000001:19 v 139'535, mlcod=137'532)", 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 5.5263767000000001, 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021832689999999998, 2024-09-17T22:59:31.478 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878212+0000", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878220+0000", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9019999999999999e-06 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878240+0000", 2024-09-17T22:59:31.480 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.984e-05 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878248+0000", 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0839999999999992e-06 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878316+0000", 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392999999999994e-05 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.481 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878346+0000", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9958999999999999e-05 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880345+0000", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001999228 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880394+0000", 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8986999999999997e-05 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.482 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1614 3.0 e140/126 3:01b2b794:::rbd_data.118894cf7479.0000000000000031:18 v 140'370, mlcod=137'366)", 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.5323496859999999, 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018540049999999999, 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.483 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.484 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5199999999999998e-07 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872244+0000", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872263+0000", 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9357000000000001e-05 2024-09-17T22:59:31.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872272+0000", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1710000000000008e-06 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872343+0000", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0766999999999995e-05 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872369+0000", 2024-09-17T22:59:31.486 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5870000000000001e-05 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874055+0000", 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001685576 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874092+0000", 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7453999999999997e-05 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1617 3.0 e140/126 3:05327ecb:::rbd_data.118894cf7479.0000000000000008:18 v 140'372, mlcod=137'366)", 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.5313400870000002, 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001099488, 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2699999999999998e-07 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.489 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873255+0000", 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3459999999999999e-06 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873265+0000", 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873279+0000", 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.452e-05 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.490 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873327+0000", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7261e-05 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873344+0000", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7703999999999999e-05 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874309+0000", 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00096465899999999998 2024-09-17T22:59:31.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874347+0000", 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8130000000000003e-05 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1618 3.4 e140/123 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:18 v 140'538, mlcod=139'535)", 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 4.5285546410000004, 2024-09-17T22:59:31.492 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013940280000000001, 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:31.493 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876034+0000", 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9999999999999997e-07 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876040+0000", 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2020000000000001e-06 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:31.494 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876049+0000", 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.799e-06 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876055+0000", 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1589999999999998e-06 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876103+0000", 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7979999999999998e-05 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.495 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876123+0000", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9817e-05 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877387+0000", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001264041 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877427+0000", 2024-09-17T22:59:31.496 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0330000000000002e-05 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:31.497 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:31.498 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:31.499 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:31.496+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:31.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:31.599 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:31.596+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:31.645 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:31.645 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:31.645 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:31.654 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:31.660 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:31.660 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:31.678 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:31.678 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:31.694 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:31.695 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:31.700 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:31.697+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:31.711 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:31.711 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:31.728 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:31.728 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:31.745 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:31.745 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:31.758 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:31.759 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:31.761 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:31.762 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:31.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:31.779 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:31.779 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:31.831 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:31.829+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:31.881 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:31.881 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:31.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:31.901 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:31.898+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:32.001 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:31.998+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:32.042 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.042 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:32.103 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:32.100+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:32.154 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.154 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:32.203 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:32.201+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:32.259 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 6, 7, 5, 2] 2024-09-17T22:59:32.259 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:59:32.259 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T22:59:32.259 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:59:32.259 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:59:32.260 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T22:59:32.267 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.267 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.276 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:32.304 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:32.301+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:32.365 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:32.365 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:59:32.365 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:32.404 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:32.401+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:32.421 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.422 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.431 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:32.506 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:32.503+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:32.534 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.534 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.544 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:32.606 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:32.603+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:32.648 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.648 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.656 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:32.707 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:32.704+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.777 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:32.819 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:32.816+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:32.906 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:59:32.909 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:32.907+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:32.927 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:32.927 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:32.937 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:33.009 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:33.006+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:33.049 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.050 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.059 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:33.109 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:33.107+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:33.163 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.163 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.172 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:33.210 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:33.207+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:33.276 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.277 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.285 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:33.329 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:33.326+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:33.388 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.388 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.396 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:33.437 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:33.434+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:33.513 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:33.510+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:33.540 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.540 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.549 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:33.614 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:33.611+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:33.651 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.651 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:33.715 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:33.713+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:33.792 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:33.803 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.803 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.812 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:33.855 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:33.852+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:33.906 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:59:33.918 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:33.915+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:33.923 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:33.923 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:33.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:33.943 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:33.943 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:33.963 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:33.963 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:33.981 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:33.981 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:34.002 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:34.002 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:34.018 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:34.015+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:34.020 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:34.021 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:34.040 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:34.041 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:34.055 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:34.055 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:34.072 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:34.072 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:34.084 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:34.085 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:34.085 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:34.085 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:34.085 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:34.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:34.135 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:34.132+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:34.221 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:34.219+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:34.241 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:34.241 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:34.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:34.241 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:34.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.1f 106/105 [PushOp(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a, version: 34'122, data_included: [0~49152,81920~163840,278528~163840,475136~163840,671744~163840,868352~163840,1512747~24346,24805157~29192], data_size: 921890, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f9e21495:::rbd_data.1188b0ea165c.0000000000000002:a@34'122, size: 29360128, copy_subset: [(0, 29360128)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 29360128, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.485356080000003, 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022360850000000001, 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.242 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.244 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:34.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:34.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.752123+0000", 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753035+0000", 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091206999999999996 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753062+0000", 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6344000000000001e-05 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753085+0000", 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3184e-05 2024-09-17T22:59:34.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753163+0000", 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7713999999999995e-05 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.753171+0000", 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7960000000000001e-06 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:53.754359+0000", 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011873229999999999 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.026186068000001, 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.251 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:34.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:34.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:34.255 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.256 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.021076983999997, 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:34.257 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:34.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.16 107/105 [PushOp(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b, version: 81'1092, data_included: [32768~196608,262144~196608,491520~196608,720896~196608,950272~98304,3145728~32768,3342336~65536,3571712~36671,3620864~16384,3801088~12481], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:68b648ff:::rbd_data.1188f7e092e0.0000000000000000:2b@81'1092, size: 15860654, copy_subset: [(0, 15860654)], clone_subset: {}, snapset: 2b=[]:{2b: [2b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3813569, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:34.262 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.877157081999997, 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022678809999999998, 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.360322+0000", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9200000000000001e-07 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361145+0000", 2024-09-17T22:59:34.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00082287500000000004 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361162+0000", 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6957999999999999e-05 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361191+0000", 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9450000000000001e-05 2024-09-17T22:59:34.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361286+0000", 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.4221000000000005e-05 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.361302+0000", 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6333000000000001e-05 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.362589+0000", 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0012874519999999999 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 39.754439724999997, 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:34.267 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.271 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 36.364598716000003, 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:34.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:34.278 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.325985703000001, 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:34.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:34.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.286 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:34.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:34.289 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 30.324219775, 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.290 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:34.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.295 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:34.296 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 30.318783477, 2024-09-17T22:59:34.297 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.298 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:34.299 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.300 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:34.301 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:34.302 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.294082992, 2024-09-17T22:59:34.303 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:34.304 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.305 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.306 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.307 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:34.308 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.289883933999999, 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.309 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:34.310 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.311 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:34.312 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:34.313 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:34.314 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.294144897999999, 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.315 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.316 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.317 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:34.318 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:34.319 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(client.4488.0:199116 3.17 e121/108 3:ed33c955:::rbd_header.1188b9dbd70:head v 121'628, mlcod=120'626)", 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.293491253999999, 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00221332, 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.320 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943987+0000", 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.321 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943988+0000", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5700000000000001e-07 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943995+0000", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1239999999999999e-06 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944093+0000", 2024-09-17T22:59:34.322 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.7671999999999996e-05 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944104+0000", 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.082e-05 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944212+0000", 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000108087 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.323 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.944237+0000", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4841000000000002e-05 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946157+0000", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0019206939999999999 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946201+0000", 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3225e-05 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.324 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 26.170781724000001, 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.325 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.327 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:34.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200096 3.11 3:8e35b8be:::rbd_data.11884120dce1.000000000000000f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.152067571, 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022879160000000001, 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200096 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.330 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.331 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085412+0000", 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0559999999999999e-06 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085435+0000", 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3000999999999999e-05 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.332 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085451+0000", 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5899000000000001e-05 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085459+0000", 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3990000000000004e-06 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.333 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085566+0000", 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085717+0000", 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015122599999999999 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085826+0000", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010882699999999999 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.086549+0000", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072324900000000003 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087552+0000", 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010025909999999999 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.335 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087652+0000", 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010069300000000001 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087699+0000", 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6470000000000001e-05 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.336 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200163 3.11 3:8ca7bd04:::rbd_header.1188b5c0f048:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.104832112, 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022726589999999998, 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200163 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.337 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:34.338 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132649+0000", 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132654+0000", 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.339 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132657+0000", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.106e-06 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132891+0000", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000234212 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132953+0000", 2024-09-17T22:59:34.340 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2217000000000004e-05 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133128+0000", 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017510200000000001 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133539+0000", 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041042999999999999 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.341 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134734+0000", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011958190000000001 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134863+0000", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128555 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134919+0000", 2024-09-17T22:59:34.342 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6168999999999997e-05 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200744 3.11 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2937740370000004, 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002277519, 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.343 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200744 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.344 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.345 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943716+0000", 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0552e-05 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943729+0000", 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2553e-05 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943736+0000", 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3350000000000001e-06 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.346 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943784+0000", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7678e-05 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943862+0000", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288999999999995e-05 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943933+0000", 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0761999999999998e-05 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.347 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944996+0000", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001062849 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945883+0000", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088707900000000004 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945939+0000", 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6242999999999997e-05 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.348 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945982+0000", 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200745 3.11 3:8e06f6ab:::rbd_data.1188d23b4ac7.0000000000000013:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 9.2937244870000004, 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002340698, 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.349 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200745 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.350 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943755+0000", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943758+0000", 2024-09-17T22:59:34.351 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0280000000000001e-06 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943765+0000", 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0929999999999997e-06 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943771+0000", 2024-09-17T22:59:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9179999999999999e-06 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944323+0000", 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055252999999999997 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944465+0000", 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014123799999999999 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.353 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944593+0000", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012883099999999999 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945529+0000", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093524199999999997 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946020+0000", 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00049166900000000003 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.354 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946051+0000", 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0681e-05 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946095+0000", 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3897999999999999e-05 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.355 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:201738 3.5 3:a2753f7f:::rbd_data.11883d082c32.0000000000000006:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:30.958332+0000", 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.279146291, 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022208309999999999, 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 201738 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.356 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958332+0000", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958332+0000", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958333+0000", 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5799999999999998e-07 2024-09-17T22:59:34.357 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958337+0000", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1230000000000002e-06 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958348+0000", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0719e-05 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958355+0000", 2024-09-17T22:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2570000000000001e-06 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958415+0000", 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9700000000000001e-05 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958502+0000", 2024-09-17T22:59:34.359 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7219999999999995e-05 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958562+0000", 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9565999999999999e-05 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.360 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.959490+0000", 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00092829399999999997 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.960460+0000", 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00097034400000000005 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.960512+0000", 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1848999999999998e-05 2024-09-17T22:59:34.361 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.960553+0000", 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0901e-05 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.362 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:201739 3.d 3:b1d2de87:::rbd_data.11883d082c32.0000000000000007:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 3.2791044830000002, 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002892113, 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 201739 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:34.363 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958375+0000", 2024-09-17T22:59:34.364 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4300000000000003e-07 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958377+0000", 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958383+0000", 2024-09-17T22:59:34.365 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8660000000000002e-06 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958388+0000", 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3550000000000004e-06 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958438+0000", 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9636999999999999e-05 2024-09-17T22:59:34.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958509+0000", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1336999999999998e-05 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958565+0000", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6437000000000002e-05 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.959484+0000", 2024-09-17T22:59:34.367 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091845700000000002 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.960546+0000", 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00106214 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.961218+0000", 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00067234900000000004 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:34.368 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.961266+0000", 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7880999999999998e-05 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:34.369 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:34.371 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:34.371 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:34.368+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:34.471 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:34.469+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:34.475 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:34.475 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:34.475 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:34.475 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:34.484 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:34.572 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:34.570+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:34.630 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:34.630 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:34.630 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:34.630 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:34.630 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:34.638 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:34.682 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:34.680+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:34.743 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:34.743 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:34.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:34.743 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:34.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26, version: 76'382, data_included: [0~631961,1179648~65536,4194304~131072], data_size: 828569, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26@76'382, size: 7340032, copy_subset: [(0, 7340032)], clone_subset: {}, snapset: 26=[]:{26: [26]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7340032, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.785133967, 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021080690000000002, 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954950+0000", 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955691+0000", 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074070899999999999 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955710+0000", 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9463999999999999e-05 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955731+0000", 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0437000000000001e-05 2024-09-17T22:59:34.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955812+0000", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1221e-05 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955822+0000", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0016e-05 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.957057+0000", 2024-09-17T22:59:34.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012356509999999999 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.755552695999999, 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001379094, 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3799999999999997e-07 2024-09-17T22:59:34.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985075+0000", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054357699999999995 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985092+0000", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7041000000000002e-05 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985103+0000", 2024-09-17T22:59:34.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0740999999999999e-05 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985162+0000", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9290999999999999e-05 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985170+0000", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7579999999999992e-06 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985910+0000", 2024-09-17T22:59:34.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074014800000000004 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.753210942999999, 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013103189999999999, 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986873+0000", 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:59:34.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987326+0000", 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00045334100000000001 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987343+0000", 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6495e-05 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987354+0000", 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0791999999999999e-05 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987411+0000", 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7445000000000003e-05 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987419+0000", 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0139999999999998e-06 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.988183+0000", 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00076374600000000004 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35, version: 91'439, data_included: [12288~28672,102400~28672,143360~28672,233472~28672,274432~28672,364544~28672,405504~28672,495616~28672,536576~28672,626688~28672,667648~28672,757760~28672,798720~28672,888832~28672,929792~20480,1019904~28672], data_size: 450560, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35@91'439, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 35=[]:{35: [35]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.736445163999999, 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017163370000000001, 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.756 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003639+0000", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.71e-07 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003943+0000", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00030434599999999999 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003969+0000", 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5700000000000001e-05 2024-09-17T22:59:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003994+0000", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4416000000000001e-05 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004076+0000", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2137000000000003e-05 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004086+0000", 2024-09-17T22:59:34.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0271999999999999e-05 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.005355+0000", 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001268595 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1528 3.4 e124/123 3:2363738a:::rbd_data.1188b9dbd70.0000000000000032:23 v 124'482, mlcod=115'479)", 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:34.760 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.685955540999998, 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022419990000000002, 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054135+0000", 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2750000000000001e-06 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054146+0000", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0829000000000001e-05 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054153+0000", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0099999999999998e-06 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054191+0000", 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8550999999999997e-05 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054205+0000", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3891e-05 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056337+0000", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021323190000000001 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056370+0000", 2024-09-17T22:59:34.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2455e-05 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1529 3.4 e124/123 3:220bdc83:::rbd_data.1188f4e2c696.000000000000010a:24 v 124'484, mlcod=115'480)", 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.682877306999998, 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025955399999999999, 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057207+0000", 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7899999999999998e-07 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057212+0000", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0479999999999998e-06 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057221+0000", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057227+0000", 2024-09-17T22:59:34.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3369999999999996e-06 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057277+0000", 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9292000000000002e-05 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057290+0000", 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059759+0000", 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0024690379999999998 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059802+0000", 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2106e-05 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1530 3.4 e124/123 3:222138c9:::rbd_data.1188f4e2c696.000000000000003e:24 v 124'486, mlcod=115'480)", 2024-09-17T22:59:34.769 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.682387835, 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023558860000000002, 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057703+0000", 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8859999999999998e-06 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057711+0000", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2949999999999994e-06 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057717+0000", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4779999999999999e-06 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057755+0000", 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8606000000000003e-05 2024-09-17T22:59:34.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057767+0000", 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1484999999999999e-05 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060021+0000", 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022539080000000002 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060051+0000", 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0679000000000002e-05 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1531 3.4 e124/123 3:2668e2f4:::rbd_data.1188f4e2c696.0000000000000042:24 v 124'488, mlcod=124'484)", 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.679645655000002, 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001073601, 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.774 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060438+0000", 2024-09-17T22:59:34.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060444+0000", 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.237e-06 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060454+0000", 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0824e-05 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060461+0000", 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060501+0000", 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0577000000000001e-05 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060515+0000", 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3429e-05 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061462+0000", 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00094694600000000005 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.061511+0000", 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9435999999999999e-05 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1534 3.4 e124/123 3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30 v 124'494, mlcod=124'490)", 2024-09-17T22:59:34.778 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.669722635999999, 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001260253, 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.68e-07 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070367+0000", 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8780000000000003e-06 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070377+0000", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7440000000000002e-06 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070384+0000", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9140000000000002e-06 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070425+0000", 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0800999999999998e-05 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070446+0000", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1603e-05 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071578+0000", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001131878 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071621+0000", 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2766999999999999e-05 2024-09-17T22:59:34.782 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1539 3.4 e124/123 3:263d8630:::rbd_data.11881ee9ab63.000000000000006a:36 v 124'504, mlcod=124'498)", 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.660637543, 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001105275, 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079446+0000", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079447+0000", 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.06e-07 2024-09-17T22:59:34.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079453+0000", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8120000000000002e-06 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079461+0000", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1699999999999997e-06 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079468+0000", 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5570000000000001e-06 2024-09-17T22:59:34.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079506+0000", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7540999999999999e-05 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.079530+0000", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4508999999999999e-05 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080522+0000", 2024-09-17T22:59:34.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099224300000000007 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.080551+0000", 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8736999999999999e-05 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.11 125/124 [PushOp(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head, version: 48'115, data_included: [0~262144,524288~287322,819200~49152,942080~49152], data_size: 647770, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head@48'115, size: 1356976, copy_subset: [(0, 1356976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1356976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:34.787 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 22.605246653000002, 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002060559, 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:34.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135765+0000", 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00092785999999999997 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135788+0000", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2623e-05 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135808+0000", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-05 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135897+0000", 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8865999999999999e-05 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135909+0000", 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2482e-05 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.136897+0000", 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098805099999999999 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200097 3.4 e130/123 3:26073371:::rbd_data.11884120dce1.0000000000000013:head v 130'509, mlcod=124'506)", 2024-09-17T22:59:34.791 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.654219483999999, 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011580830000000001, 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085864+0000", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3599999999999999e-07 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085870+0000", 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3940000000000004e-06 2024-09-17T22:59:34.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085888+0000", 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8505999999999999e-05 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085897+0000", 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3549999999999994e-06 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085978+0000", 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0677e-05 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.085999+0000", 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1192e-05 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.086969+0000", 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00096941300000000002 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.087022+0000", 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.321e-05 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200163 3.11 e130/126 3:8ca7bd04:::rbd_header.1188b5c0f048:head v 130'473, mlcod=130'471)", 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.606860906000001, 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011031330000000001, 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.796 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133222+0000", 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133223+0000", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1500000000000004e-07 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133230+0000", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.477e-06 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133240+0000", 2024-09-17T22:59:34.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133246+0000", 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9839999999999998e-06 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133292+0000", 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6783999999999998e-05 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.133311+0000", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8919e-05 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.134295+0000", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000983252 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.134326+0000", 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1061e-05 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.800 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPushReply(3.1f 130/129 [PushReplyOp(3:f89167a1:::rbd_data.118875333e6d.0000000000000000:f)])", 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.487450007, 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0011045429999999999, 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.801 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252633+0000", 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252634+0000", 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7400000000000002e-07 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252639+0000", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9760000000000003e-06 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252652+0000", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2744e-05 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252663+0000", 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0869999999999999e-05 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.252716+0000", 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2947e-05 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:17.253738+0000", 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0010222320000000001 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.804 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:200744 3.11 e138/126 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head v 138'496, mlcod=136'494)", 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.7955959640000003, 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001254745, 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944487+0000", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944488+0000", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7799999999999997e-07 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944494+0000", 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0290000000000002e-06 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944510+0000", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6548000000000001e-05 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944520+0000", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2830000000000007e-06 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944611+0000", 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1547999999999998e-05 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.944633+0000", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1599000000000001e-05 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.945699+0000", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001066333 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:24.945742+0000", 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3127000000000002e-05 2024-09-17T22:59:34.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1611 3.0 e139/126 3:0562d798:::rbd_data.1188457bc75a.000000000000000d:19 v 139'368, mlcod=130'365)", 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.8626457859999999, 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0014449560000000001, 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-07 2024-09-17T22:59:34.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877444+0000", 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1809999999999998e-06 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877488+0000", 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3646000000000002e-05 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877497+0000", 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9919999999999996e-06 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877600+0000", 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010299800000000001 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877624+0000", 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4329e-05 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878837+0000", 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001212631 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878883+0000", 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5911000000000001e-05 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.813 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1612 3.4 e139/123 3:268a8c58:::rbd_data.1188457bc75a.0000000000000001:19 v 139'535, mlcod=137'532)", 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 8.8618723910000003, 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021832689999999998, 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.814 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878212+0000", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878220+0000", 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9019999999999999e-06 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878240+0000", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.984e-05 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878248+0000", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0839999999999992e-06 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878316+0000", 2024-09-17T22:59:34.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392999999999994e-05 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878346+0000", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9958999999999999e-05 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880345+0000", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001999228 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880394+0000", 2024-09-17T22:59:34.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8986999999999997e-05 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1614 3.0 e140/126 3:01b2b794:::rbd_data.118894cf7479.0000000000000031:18 v 140'370, mlcod=137'366)", 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 7.8678453770000001, 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018540049999999999, 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5199999999999998e-07 2024-09-17T22:59:34.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872244+0000", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872263+0000", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9357000000000001e-05 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872272+0000", 2024-09-17T22:59:34.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1710000000000008e-06 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872343+0000", 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0766999999999995e-05 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872369+0000", 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5870000000000001e-05 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874055+0000", 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001685576 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874092+0000", 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7453999999999997e-05 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1617 3.0 e140/126 3:05327ecb:::rbd_data.118894cf7479.0000000000000008:18 v 140'372, mlcod=137'366)", 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:34.822 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 7.8668357780000004, 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001099488, 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:34.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873248+0000", 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2699999999999998e-07 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873255+0000", 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3459999999999999e-06 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873265+0000", 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873279+0000", 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.452e-05 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873327+0000", 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7261e-05 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.873344+0000", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7703999999999999e-05 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874309+0000", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00096465899999999998 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874347+0000", 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8130000000000003e-05 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.826 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1618 3.4 e140/123 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:18 v 140'538, mlcod=139'535)", 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 7.8640503319999997, 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013940280000000001, 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:34.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876034+0000", 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9999999999999997e-07 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876040+0000", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2020000000000001e-06 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876049+0000", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.799e-06 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876055+0000", 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1589999999999998e-06 2024-09-17T22:59:34.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876103+0000", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7979999999999998e-05 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876123+0000", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9817e-05 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877387+0000", 2024-09-17T22:59:34.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001264041 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877427+0000", 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0330000000000002e-05 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:34.831 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:34.833 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:34.833 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:34.830+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:34.933 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:34.931+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:34.989 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:34.989 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:34.990 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:34.990 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:34.999 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:35.041 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:35.038+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:35.136 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:35.133+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:35.143 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.143 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:35.143 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:35.143 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:35.143 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:35.236 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:35.234+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:35.256 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:35.256 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 118.987553069, 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.257 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.259 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:35.260 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.261 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 110.80991407400001, 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:35.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.263 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:35.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.266 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.267 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165723826999994, 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.268 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.269 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:35.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:35.271 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.272 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165701197999994, 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.273 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.274 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:35.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:35.276 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.277 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165501112000001, 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.278 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.279 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:35.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165475540000003, 2024-09-17T22:59:35.282 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:35.286 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165294489000004, 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.287 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:35.289 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.290 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:35.291 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165259059999997, 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.293 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:35.295 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.165160158999996, 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.296 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:35.298 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:35.299 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:35.300 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.164983999, 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.301 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.302 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:35.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.304 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.164924502999995, 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.305 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:35.306 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.307 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.309 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.154651333999993, 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:35.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.312 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.313 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.154563158000002, 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.314 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.315 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:35.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.317 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.154510178999999, 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:35.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.320 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.321 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.322 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.154452215000006, 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.324 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.325 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:35.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.327 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.154167928999996, 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:35.329 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.330 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.331 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.153991746000003, 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.332 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.333 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.334 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:35.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.336 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.153312608999997, 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.337 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.338 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.339 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:35.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.153289134999994, 2024-09-17T22:59:35.341 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.343 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:35.345 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:200307 3.1 3:85117544:::rbd_object_map.1188d79d2447:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.363751907999999, 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.98269046800000004, 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.346 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 200307 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889649+0000", 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3500000000000005e-07 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889658+0000", 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0410000000000003e-06 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889668+0000", 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9979999999999998e-06 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889673+0000", 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1109999999999999e-06 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889718+0000", 2024-09-17T22:59:35.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5235999999999999e-05 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889724+0000", 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8130000000000001e-06 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.869697+0000", 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.97997323999999997 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.869872+0000", 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000174895 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.870085+0000", 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021357599999999999 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.871356+0000", 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001270925 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872044+0000", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00068763899999999998 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872296+0000", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000251914 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872338+0000", 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2245e-05 2024-09-17T22:59:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.353 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.353 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.353 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.353 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.353 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.354 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:35.355 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:35.352+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:35.455 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:35.452+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:35.498 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.498 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:35.498 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:35.498 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.508 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:35.555 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:35.553+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:35.651 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.651 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:35.651 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:35.651 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:35.651 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.657 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:35.654+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:35.659 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:35.758 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:35.756+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722872787, 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:35.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:35.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722852372000006, 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.769 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:35.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:35.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:35.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722831313, 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:35.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.780 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722808208000004, 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:35.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:35.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722785152, 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:35.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:35.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722754519000006, 2024-09-17T22:59:35.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:35.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722730542999997, 2024-09-17T22:59:35.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:35.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:35.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.799 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.800 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722700364999994, 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:35.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:35.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722681387999998, 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.806 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:35.807 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:35.808 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.811 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.722242476999995, 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.813 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.816 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:35.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:35.819 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.721953872, 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.820 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.821 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:35.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.721326968, 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.825 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.826 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:35.827 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.828 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.829 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.721292528999996, 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.830 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.831 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.832 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:35.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.720904368999996, 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.834 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.835 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:35.836 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.838 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:35.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.719124237000003, 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.841 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:35.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:35.843 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.844 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.719109969000002, 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.845 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.846 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.847 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:35.848 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.719099513000003, 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:35.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:35.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:35.852 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:35.853 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.719081231999994, 2024-09-17T22:59:35.854 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:35.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.857 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.718661764000004, 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.860 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:35.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:35.862 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 79.718035518999997, 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:35.863 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:35.865 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:35.866 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:35.867 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:35.868 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.869 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:35.869 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:35.865+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:35.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:59:35.970 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:35.968+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:35.983 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:35.983 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:35.983 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:35.983 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:35.993 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:36.077 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:36.075+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:36.084 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:36.143 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:36.165 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:36.165 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:36.166 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:36.171 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:36.169+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:36.210 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:36.211 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:36.229 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:36.230 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:36.246 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:36.246 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:36.262 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:36.262 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:36.272 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:36.269+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:36.276 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.276 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.279 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:36.279 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:36.285 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:36.296 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:36.296 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:36.312 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:36.313 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:36.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:36.329 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:36.373 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:36.370+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:36.390 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.390 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.398 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:36.474 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:36.472+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:36.540 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.540 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.548 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:36.592 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:36.589+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:36.653 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.653 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.662 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:36.705 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:36.701+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:36.777 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:36.775+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:36.807 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.807 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.816 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:36.878 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:36.876+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:36.930 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:36.966 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:36.966 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:36.976 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:36.986 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:36.984+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:37.080 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:37.078+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:37.088 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.088 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:37.182 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:37.180+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:37.242 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.242 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.250 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:37.282 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:37.280+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:37.375 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 6, 7, 5, 2] 2024-09-17T22:59:37.375 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:59:37.375 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T22:59:37.375 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:59:37.375 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:59:37.375 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T22:59:37.393 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.393 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.394 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:37.391+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:37.403 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:37.482 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:37.482 INFO:teuthology.orchestra.run.smithi171.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:59:37.482 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:37.484 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:37.482+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:37.548 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.549 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.557 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:37.585 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:37.583+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:37.659 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.660 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.668 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:37.686 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:37.684+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:37.772 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.772 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.781 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:37.826 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:37.824+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:37.884 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:37.884 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:37.887 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:37.885+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:37.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:37.988 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:37.986+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:38.021 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:38.022 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:38.033 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:38.118 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:38.116+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:38.179 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:38.179 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:38.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:38.229 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:38.226+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:38.291 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:38.289+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:38.331 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:38.331 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:38.340 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:38.340 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:38.392 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:38.389+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:38.451 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:38.451 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:38.461 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:38.468 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:38.468 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:38.485 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:38.485 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:38.493 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:38.491+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:38.501 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:38.502 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:38.518 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:38.518 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:38.535 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:38.535 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:38.553 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:38.554 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:38.568 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:38.569 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:38.573 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:38.573 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:38.573 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:38.573 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:38.573 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:38.583 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:38.585 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:38.585 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:38.594 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:38.592+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:38.689 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:16191 3.1f 3.a928479f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 122.42069001199999, 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053350547999999998, 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:38.690 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 16191 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.265847+0000", 2024-09-17T22:59:38.691 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267354+0000", 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001507167 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267359+0000", 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1599999999999997e-06 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267371+0000", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2615e-05 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.267378+0000", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9949999999999999e-06 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319189+0000", 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051810186000000001 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:36.319197+0000", 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4249999999999998e-06 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:23365 3.1f 3:fc202f96:::rbd_data.1188aa1637fb.00000000000000b6:head [set-alloc-hint object_size 1048576 write_size 1048576,write 123296~5659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 114.243051017, 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.641013568, 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.694 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 23365 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443486+0000", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443487+0000", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.688e-06 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443498+0000", 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0491e-05 2024-09-17T22:59:38.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443513+0000", 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5784e-05 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443521+0000", 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5379999999999997e-06 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443574+0000", 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3451e-05 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443676+0000", 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010158300000000001 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.443771+0000", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5152e-05 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:44.444361+0000", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058940900000000005 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789437+0000", 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.345076559 2024-09-17T22:59:38.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:45.789453+0000", 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5906999999999999e-05 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084407+0000", 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.29495424199999998 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.700 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:46.084499+0000", 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1763999999999998e-05 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78679 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 556032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598860770000002, 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053438477999999998, 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.701 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78679 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.702 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087676+0000", 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087678+0000", 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5990000000000001e-06 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087683+0000", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087687+0000", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0709999999999996e-06 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.140950+0000", 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05326314 2024-09-17T22:59:38.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141023+0000", 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3110999999999998e-05 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141114+0000", 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0818999999999999e-05 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78681 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 568320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598838141000002, 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053687117999999999, 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78681 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.706 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087698+0000", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087699+0000", 2024-09-17T22:59:38.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087701+0000", 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0260000000000002e-06 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087707+0000", 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0460000000000002e-06 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.708 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087711+0000", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2289999999999997e-06 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141172+0000", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053460979999999998 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141249+0000", 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6884000000000005e-05 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141386+0000", 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000136568 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78683 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 580608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598638054999995, 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053721269000000002, 2024-09-17T22:59:38.710 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78683 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087898+0000", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087899+0000", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5600000000000005e-07 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087902+0000", 2024-09-17T22:59:38.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0340000000000001e-06 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087910+0000", 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087916+0000", 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.713 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141429+0000", 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053513717000000002 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141520+0000", 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0439999999999995e-05 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141620+0000", 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9866999999999998e-05 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.714 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78685 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598612482999997, 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053942834000000002, 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78685 2024-09-17T22:59:38.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087924+0000", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2800000000000003e-07 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087926+0000", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087933+0000", 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5050000000000004e-06 2024-09-17T22:59:38.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.087937+0000", 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2259999999999999e-06 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141664+0000", 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053726968 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.718 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141746+0000", 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2170999999999996e-05 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141867+0000", 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000120896 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78687 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 605184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.719 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598431431999998, 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0539414, 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78687 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088105+0000", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088106+0000", 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:38.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088110+0000", 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.929e-06 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088122+0000", 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1960000000000001e-05 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088128+0000", 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5270000000000004e-06 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141887+0000", 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053758902999999997 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.141940+0000", 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2290000000000002e-05 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142047+0000", 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000106915 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78689 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 617472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598396003000005, 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054106640999999997, 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.724 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78689 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088141+0000", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.51e-07 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088143+0000", 2024-09-17T22:59:38.726 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088147+0000", 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5750000000000002e-06 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088150+0000", 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.727 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142073+0000", 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053922732000000001 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142148+0000", 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5128000000000003e-05 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142247+0000", 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8937999999999994e-05 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.728 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78691 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 629760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598297102000004, 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054232850999999999, 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.729 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78691 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088239+0000", 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088240+0000", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4799999999999998e-07 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088243+0000", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088255+0000", 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1829e-05 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088263+0000", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9330000000000001e-06 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142291+0000", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054027770000000003 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142374+0000", 2024-09-17T22:59:38.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2987999999999999e-05 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142472+0000", 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.8562999999999998e-05 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78693 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 642048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:38.733 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598120941999994, 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054261411000000002, 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78693 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:38.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088416+0000", 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088417+0000", 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7900000000000007e-07 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.735 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088422+0000", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0100000000000003e-06 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088442+0000", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0211e-05 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088450+0000", 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8930000000000005e-06 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.736 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142499+0000", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054049511000000001 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142573+0000", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3566999999999995e-05 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142677+0000", 2024-09-17T22:59:38.737 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010424 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78695 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 654336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.598061446000003, 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.054464798000000002, 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.738 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78695 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088475+0000", 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088476+0000", 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088478+0000", 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6699999999999998e-06 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088493+0000", 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4632e-05 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.088499+0000", 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0270000000000001e-06 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142747+0000", 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0542481 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142840+0000", 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2613000000000003e-05 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.142940+0000", 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100185 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78901 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 545792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.742 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587788277000001, 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053394586000000001, 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78901 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098748+0000", 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098749+0000", 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098751+0000", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9719999999999999e-06 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098755+0000", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.121e-06 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098758+0000", 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8890000000000002e-06 2024-09-17T22:59:38.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152022+0000", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053264644999999999 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152074+0000", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2198999999999999e-05 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152143+0000", 2024-09-17T22:59:38.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8357999999999998e-05 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78903 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 558080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587700100999996, 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053466277, 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78903 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098836+0000", 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098837+0000", 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9500000000000001e-07 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098841+0000", 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098852+0000", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0592e-05 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098859+0000", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5849999999999996e-06 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152169+0000", 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053310465000000001 2024-09-17T22:59:38.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152232+0000", 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2949999999999999e-05 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152303+0000", 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0681999999999999e-05 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78905 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 570368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587647122000007, 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053668421000000001, 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78905 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.752 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:38.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098889+0000", 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.756 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098890+0000", 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7599999999999997e-07 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098893+0000", 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.207e-06 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098901+0000", 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2740000000000007e-06 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.757 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098906+0000", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152347+0000", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053441150999999999 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152438+0000", 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0832e-05 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152558+0000", 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00011976099999999999 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78907 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587589158, 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053820272000000002, 2024-09-17T22:59:38.759 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78907 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.098947+0000", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099190+0000", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00024280800000000001 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099195+0000", 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:59:38.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099206+0000", 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1296e-05 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099215+0000", 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8619999999999992e-06 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152599+0000", 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053383615000000002 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152659+0000", 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0791999999999997e-05 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152768+0000", 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000108291 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78909 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587304872000004, 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053761471999999998, 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.764 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78909 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099232+0000", 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099373+0000", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014168500000000001 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099377+0000", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8909999999999997e-06 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099387+0000", 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.4180000000000003e-06 2024-09-17T22:59:38.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099393+0000", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8579999999999996e-06 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152806+0000", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053412989000000001 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152878+0000", 2024-09-17T22:59:38.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2472999999999995e-05 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.152993+0000", 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000115158 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78911 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.587128688999996, 2024-09-17T22:59:38.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053773552000000002, 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78911 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.099408+0000", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100021+0000", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00061297999999999995 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100026+0000", 2024-09-17T22:59:38.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1630000000000004e-06 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100038+0000", 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2078e-05 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100045+0000", 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3679999999999996e-06 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153039+0000", 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052994036000000001 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153115+0000", 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153181+0000", 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6407000000000003e-05 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.772 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78915 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.586449552000005, 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053455862, 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.773 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78915 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100087+0000", 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100088+0000", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100090+0000", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100097+0000", 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.4680000000000003e-06 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100103+0000", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6300000000000003e-06 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153390+0000", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053286675999999998 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153477+0000", 2024-09-17T22:59:38.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7751999999999996e-05 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153543+0000", 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5530000000000004e-05 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:78917 3.6 3:65055b3d:::rbd_data.1188b9dbd70.0000000000000001:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.586426078000002, 2024-09-17T22:59:38.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.053643498999999997, 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 78917 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.100110+0000", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102019+0000", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019088519999999999 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102024+0000", 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7099999999999998e-06 2024-09-17T22:59:38.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102035+0000", 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1430000000000001e-05 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.102042+0000", 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.002e-06 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153585+0000", 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051542708 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153672+0000", 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6928999999999994e-05 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.153754+0000", 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1867999999999996e-05 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.781 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:200307 3.1 3:85117544:::rbd_object_map.1188d79d2447:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 17.796888850999999, 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.98269046800000004, 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 200307 2024-09-17T22:59:38.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889648+0000", 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:38.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889649+0000", 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3500000000000005e-07 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889658+0000", 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0410000000000003e-06 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889668+0000", 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9979999999999998e-06 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889673+0000", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1109999999999999e-06 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889718+0000", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5235999999999999e-05 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:20.889724+0000", 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8130000000000001e-06 2024-09-17T22:59:38.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.869697+0000", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.97997323999999997 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.869872+0000", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000174895 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.870085+0000", 2024-09-17T22:59:38.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021357599999999999 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.871356+0000", 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001270925 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872044+0000", 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00068763899999999998 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:38.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872296+0000", 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000251914 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:21.872338+0000", 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2245e-05 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:38.788 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:38.790 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:38.790 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:38.787+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:38.890 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:38.888+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:38.893 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:38.893 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:38.894 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:38.894 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:38.902 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:38.945 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:59:38.992 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:38.990+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:39.026 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:39.026 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:39.026 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:39.026 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:39.026 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:39.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:39.093 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:39.091+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.149 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77620 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 582656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.108032340999998, 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080775711, 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77620 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.150 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038079+0000", 2024-09-17T22:59:39.151 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038081+0000", 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0310000000000001e-06 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038087+0000", 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0449999999999997e-06 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.152 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038092+0000", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4600000000000002e-06 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118605+0000", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080512346999999998 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118661+0000", 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6588000000000001e-05 2024-09-17T22:59:39.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118854+0000", 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019286600000000001 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77622 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 594944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.108011926000003, 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081113346000000003, 2024-09-17T22:59:39.154 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77622 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038099+0000", 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.84e-07 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.156 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038101+0000", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.063e-06 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038108+0000", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1510000000000001e-06 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038112+0000", 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:59:39.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118896+0000", 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080784099999999998 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.118980+0000", 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4050999999999999e-05 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.158 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119212+0000", 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00023190999999999999 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77623 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 607232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107990866999998, 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428766999999999, 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.159 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77623 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.160 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.161 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038120+0000", 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038123+0000", 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.164 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038130+0000", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038136+0000", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1430000000000003e-06 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119255+0000", 2024-09-17T22:59:39.165 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081119058999999993 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119333+0000", 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7702999999999998e-05 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119549+0000", 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000215675 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77624 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 619520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107967762000001, 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081735154000000004, 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77624 2024-09-17T22:59:39.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038143+0000", 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.168 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038144+0000", 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038146+0000", 2024-09-17T22:59:39.169 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1490000000000001e-06 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038155+0000", 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0140000000000004e-06 2024-09-17T22:59:39.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038160+0000", 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2140000000000004e-06 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119589+0000", 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081428609999999998 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.171 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119674+0000", 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5302000000000004e-05 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119878+0000", 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000204453 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.172 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77625 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 631808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107944705999998, 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082035275000000005, 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77625 2024-09-17T22:59:39.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038166+0000", 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.174 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038167+0000", 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038178+0000", 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1306e-05 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038185+0000", 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.3339999999999996e-06 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.175 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038190+0000", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9710000000000003e-06 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.119922+0000", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081732025999999999 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120009+0000", 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6976000000000002e-05 2024-09-17T22:59:39.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120201+0000", 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00019224300000000001 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77626 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 644096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107914073000003, 2024-09-17T22:59:39.177 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082326315999999997, 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77626 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:39.178 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038197+0000", 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.179 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038199+0000", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038209+0000", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.9280000000000005e-06 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038214+0000", 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9960000000000001e-06 2024-09-17T22:59:39.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120243+0000", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082028948000000004 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120307+0000", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3275000000000007e-05 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120523+0000", 2024-09-17T22:59:39.181 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021652800000000001 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77650 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 6144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107890096999995, 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082463048999999997, 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.182 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77650 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:39.183 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038221+0000", 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038231+0000", 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3370000000000006e-06 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.184 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038239+0000", 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3189999999999995e-06 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038244+0000", 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2490000000000001e-06 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120558+0000", 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082313596000000003 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.185 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120611+0000", 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3256e-05 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120684+0000", 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2881999999999998e-05 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.186 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77653 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107859919000006, 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082582213000000002, 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77653 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.187 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.188 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038251+0000", 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7700000000000005e-07 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038253+0000", 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038259+0000", 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9719999999999997e-06 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.189 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038264+0000", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8779999999999997e-06 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120721+0000", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082457111999999999 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120768+0000", 2024-09-17T22:59:39.190 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6292999999999999e-05 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120833+0000", 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5462000000000004e-05 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77655 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:39.191 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107840941999996, 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082802634, 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77655 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:39.192 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038270+0000", 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038672+0000", 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00040172999999999999 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.193 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038676+0000", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7210000000000002e-06 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038688+0000", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1793999999999999e-05 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038696+0000", 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.5989999999999996e-06 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.194 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120872+0000", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082176656000000001 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.120959+0000", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6897999999999994e-05 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121073+0000", 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000113236 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.195 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77657 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107402031000007, 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082585240000000004, 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.196 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77657 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038709+0000", 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038963+0000", 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00025429599999999999 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038967+0000", 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6550000000000002e-06 2024-09-17T22:59:39.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038981+0000", 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4106e-05 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038987+0000", 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1990000000000003e-06 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.199 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121124+0000", 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.08213703 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121206+0000", 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1475000000000002e-05 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121294+0000", 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8479000000000004e-05 2024-09-17T22:59:39.200 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77659 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.107113425999998, 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082446725999999998, 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.201 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77659 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.038997+0000", 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039593+0000", 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00059556799999999999 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039597+0000", 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3549999999999998e-06 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.203 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039606+0000", 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.0010000000000007e-06 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039613+0000", 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0450000000000003e-06 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121335+0000", 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081721243999999998 2024-09-17T22:59:39.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121383+0000", 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8267999999999999e-05 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121444+0000", 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1245000000000001e-05 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77661 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.106486521999997, 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081980979999999995, 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77661 2024-09-17T22:59:39.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.207 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039624+0000", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039625+0000", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039627+0000", 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:59:39.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039645+0000", 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7742000000000001e-05 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039651+0000", 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.156e-06 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.209 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121479+0000", 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081827724000000004 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121545+0000", 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6434999999999997e-05 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121605+0000", 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9886000000000002e-05 2024-09-17T22:59:39.210 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77663 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.106452082999994, 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.082150886000000006, 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.211 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77663 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.039659+0000", 2024-09-17T22:59:39.212 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040020+0000", 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00036120800000000002 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040024+0000", 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0260000000000001e-06 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040034+0000", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5060000000000001e-06 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040038+0000", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3279999999999999e-06 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121632+0000", 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081593900999999996 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.214 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121700+0000", 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8595000000000003e-05 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121810+0000", 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010932199999999999 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77665 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.106063922999994, 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081996022000000002, 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77665 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.216 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.040047+0000", 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.217 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041737+0000", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0016899630000000001 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041742+0000", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.826e-06 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041751+0000", 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7049999999999994e-06 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.218 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041758+0000", 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9469999999999997e-06 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121853+0000", 2024-09-17T22:59:39.219 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080094422999999998 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.121933+0000", 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0209000000000004e-05 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122043+0000", 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000109949 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77671 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.104283791, 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080806508999999999, 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.221 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77671 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041827+0000", 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.46e-07 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041829+0000", 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.612e-06 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.223 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041835+0000", 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5659999999999997e-06 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041838+0000", 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.914e-06 2024-09-17T22:59:39.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122490+0000", 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080652969000000005 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122572+0000", 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1132999999999997e-05 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.225 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122634+0000", 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1969000000000002e-05 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77673 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.104269522999999, 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081021195000000004, 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.226 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77673 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.227 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041841+0000", 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041842+0000", 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.228 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041843+0000", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.189e-06 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041846+0000", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.191e-06 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041849+0000", 2024-09-17T22:59:39.229 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.8629999999999999e-06 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122671+0000", 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080822056000000003 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122757+0000", 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6132999999999997e-05 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.230 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122863+0000", 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010552000000000001 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77676 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 145408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.104259067000001, 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081187147000000001, 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.231 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77676 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.233 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.233 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.233 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:39.233 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041852+0000", 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041860+0000", 2024-09-17T22:59:39.234 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2490000000000001e-06 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041864+0000", 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3979999999999999e-06 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041866+0000", 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-06 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.235 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122905+0000", 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081038652000000003 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.122970+0000", 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4974000000000005e-05 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123039+0000", 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8980999999999996e-05 2024-09-17T22:59:39.236 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77678 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 157696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.104240786000005, 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081393513000000001, 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.237 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77678 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.041870+0000", 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042254+0000", 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00038364300000000003 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.239 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042259+0000", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042270+0000", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1525999999999999e-05 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042277+0000", 2024-09-17T22:59:39.240 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0990000000000001e-06 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123078+0000", 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080800458000000006 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123163+0000", 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.5555000000000003e-05 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.241 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123264+0000", 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000100343 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77682 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.103821318000001, 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.081155130000000006, 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.242 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77682 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042290+0000", 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042873+0000", 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058311400000000003 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.244 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042877+0000", 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3540000000000002e-06 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042886+0000", 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4849999999999992e-06 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.245 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042891+0000", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7339999999999996e-06 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123302+0000", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080410957000000005 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123366+0000", 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3744999999999994e-05 2024-09-17T22:59:39.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123445+0000", 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8740999999999997e-05 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:77686 3.0 3:04deb8c3:::rbd_data.1188b9dbd70.0000000000000000:23 [read 194560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 83.103195072999995, 2024-09-17T22:59:39.247 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.080930540999999995, 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 77686 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.248 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.042916+0000", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044019+0000", 2024-09-17T22:59:39.249 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001103672 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044024+0000", 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6630000000000001e-06 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044035+0000", 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0933e-05 2024-09-17T22:59:39.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.044042+0000", 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123725+0000", 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.079682820000000001 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123782+0000", 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6898999999999999e-05 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:16.123846+0000", 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4502e-05 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.252 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:39.254 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:39.254 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:39.251+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:39.355 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:39.353+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:39.399 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:39.399 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:39.399 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:39.400 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:39.408 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:39.456 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:39.453+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:39.512 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:39.512 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:39.512 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:39.512 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:39.513 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:39.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:39.561 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:39.559+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:39.657 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:39.654+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.665 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 100.50886177700001, 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.666 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:59:39.667 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.668 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:59:39.669 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:59:39.670 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.157073378000007, 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.673 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.674 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:59:39.675 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:59:39.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.677 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.157052668000006, 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.678 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:39.679 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:59:39.680 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:59:39.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.682 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.157032873000006, 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.683 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.684 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:39.685 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.686 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:59:39.687 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.688 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.689 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.157013629999994, 2024-09-17T22:59:39.690 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.691 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:39.693 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:59:39.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.695 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.696 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.697 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:59:39.698 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:59:39.699 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.700 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156992489999993, 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.701 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.702 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.703 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:59:39.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.705 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:59:39.706 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:59:39.707 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156967401000003, 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.708 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:39.709 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.710 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.712 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156948057999998, 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:59:39.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:59:39.716 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156926034999998, 2024-09-17T22:59:39.717 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.718 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:39.719 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.721 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.722 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:59:39.723 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.724 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.725 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.726 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.726 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.726 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:39.726 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156831894000007, 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.727 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.728 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.729 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:39.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.732 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.733 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:59:39.734 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.735 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.736 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:39.736 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156805121999994, 2024-09-17T22:59:39.736 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:59:39.736 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.736 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:59:39.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.738 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.739 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.739 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.739 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:39.739 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:59:39.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.741 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.742 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.742 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:59:39.742 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:59:39.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.742 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.743 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.744 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.746 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.746 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:39.746 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156784595999994, 2024-09-17T22:59:39.746 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:59:39.746 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.747 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.747 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.747 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.748 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:39.749 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.750 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.750 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.750 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:39.750 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:59:39.751 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.751 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.751 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.751 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:59:39.751 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:59:39.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.753 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.754 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.754 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:59:39.754 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:59:39.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.754 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.756 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.756 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.756 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.756 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:39.756 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156763303000005, 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.757 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:39.758 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.760 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.760 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.760 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:59:39.760 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:59:39.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.761 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.761 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.761 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:59:39.761 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:59:39.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:59:39.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156734427999993, 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.764 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:39.766 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:59:39.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:59:39.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.769 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156693348999994, 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:59:39.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.771 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.772 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.773 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:39.774 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 86.156669325999999, 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.775 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.776 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.777 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:59:39.778 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.271926699000005, 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.779 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.780 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.781 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:59:39.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.783 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.271905071000006, 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.784 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.785 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:59:39.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.787 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.788 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.271883887000001, 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.789 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:59:39.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 82.271862807000005, 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.794 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:39.795 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:59:39.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:39.798 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:39.799 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:39.800 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:39.800 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:39.798+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:59:39.901 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:39.898+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:39.914 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:39.914 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:39.914 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:39.923 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:40.002 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:40.000+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:40.031 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:40.031 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:40.031 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:40.031 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:40.031 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:40.040 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:40.125 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:40.123+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.798114358999996, 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:40.187 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.188 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.189 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:40.190 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:40.191 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.192 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.759141407000001, 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.195 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.197 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.759120631000002, 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:40.198 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.200 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.201 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:40.202 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.203 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.759101841000003, 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.204 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.206 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.207 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:40.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.209 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.759056630000003, 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.210 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.211 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:40.212 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.213 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 82.759022802000004, 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.215 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.216 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.217 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.218 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.219 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.054105534999998, 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.220 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.221 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.222 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.223 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.054085067000003, 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.225 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.226 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:40.227 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.228 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.054063505999999, 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.229 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.230 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.232 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.233 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.054043688999997, 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.234 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.236 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:40.237 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.054023700999998, 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:40.239 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.240 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.241 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.243 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053358287999998, 2024-09-17T22:59:40.244 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.245 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.246 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.247 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:40.248 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053335543000003, 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.249 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.250 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.251 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.252 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.253 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053314409999999, 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:40.254 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.255 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.256 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:40.257 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:40.258 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:40.259 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053294293, 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.260 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:40.261 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.262 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:40.263 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.264 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053267083000001, 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.265 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.266 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:40.268 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.269 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053096222000001, 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.270 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.271 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.273 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 60.053071903999999, 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.274 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.275 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.276 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:40.277 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.278 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 55.047797901999999, 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.279 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:40.280 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.281 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:40.282 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.283 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:40.284 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.285 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200002 3.1b 3:dea2114f:::rbd_object_map.11884120dce1:head [call lock.unlock in=10b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 24.189223456000001, 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0273163620000001, 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.286 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200002 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.287 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994166+0000", 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994172+0000", 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2330000000000003e-06 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994181+0000", 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3190000000000001e-06 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994188+0000", 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8190000000000003e-06 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994239+0000", 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0865000000000002e-05 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994250+0000", 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1191e-05 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.290 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019705+0000", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0254550520000001 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019922+0000", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021627500000000001 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020180+0000", 2024-09-17T22:59:40.291 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025870299999999999 2024-09-17T22:59:40.292 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.292 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.292 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:40.292 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020983+0000", 2024-09-17T22:59:40.292 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080232199999999995 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021341+0000", 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000358356 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.293 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021391+0000", 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0390999999999999e-05 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021481+0000", 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9673999999999996e-05 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.294 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.295 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.295 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.295 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:40.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:40.297 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:40.294+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:40.397 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:40.395+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:40.440 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:40.440 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:40.441 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:40.449 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:40.498 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:40.496+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:40.551 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:40.551 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:40.551 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:40.552 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:40.552 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:40.560 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:40.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:40.604 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:40.602+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:40.665 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:40.665 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.409146795, 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.668 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:40.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40913079000001, 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:40.670 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.671 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.672 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.673 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.674 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:40.675 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40909802, 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:40.676 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.677 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.678 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.679 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.409070282, 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.680 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.681 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:40.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.683 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.684 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.409047863, 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.685 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.686 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:40.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.688 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.689 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.409025583, 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:40.690 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.691 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.692 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:40.693 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:40.694 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.408818178, 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:40.695 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.696 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.697 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.698 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:40.699 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.408271563, 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.700 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.701 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.702 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:40.703 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.407200367, 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.704 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.705 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.706 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:40.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:40.708 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.407188912, 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.709 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:40.710 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.711 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.712 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:40.713 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40717793499999, 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.714 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.715 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:40.716 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:40.717 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.718 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.405103803, 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.720 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.721 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.722 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.723 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:40.724 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.405091489, 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.725 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.726 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:40.727 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.728 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.405078252, 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.729 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.731 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.732 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.733 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40506742300001, 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.734 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.735 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:40.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:40.737 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40310854800001, 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.738 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.739 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:40.740 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:40.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.742 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.743 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.744 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:40.745 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:40.746 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.747 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40308419, 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.748 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.749 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.749 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:40.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.749 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.750 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.751 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.751 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:40.751 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.751 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.752 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.752 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:40.752 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:40.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.753 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.754 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.754 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:40.754 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:40.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.754 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.755 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.756 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:40.756 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:40.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.757 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:40.759 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40305497200001, 2024-09-17T22:59:40.760 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:40.760 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.760 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.760 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.760 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.761 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.761 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:40.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.761 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.762 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:40.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.764 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:40.765 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 124.40227530200001, 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.767 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.768 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:40.769 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 42.783416764000002, 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:40.770 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.771 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:40.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:40.773 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:40.774 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.775 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:40.776 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:40.777 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:40.778 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:40.779 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:40.779 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:40.780 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:40.780 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:40.780 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:40.781 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:40.781 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:40.781 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:40.781 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:40.776+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:40.789 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:40.789 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:40.805 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:40.805 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:40.823 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:40.823 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:40.838 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:40.838 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:40.880 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:40.878+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:40.924 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:40.924 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:40.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:41.018 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:41.016+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:41.082 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:41.079+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:41.084 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.084 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:41.183 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:41.181+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:41.237 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.238 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:41.291 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:41.289+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:41.351 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.351 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:41.400 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:41.398+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:41.486 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:41.484+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:41.504 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.504 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:41.587 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:41.585+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:41.616 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.616 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.625 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:41.688 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:41.686+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:41.732 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.732 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.741 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:41.789 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:41.787+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:41.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:59:41.890 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:41.888+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:41.894 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:41.991 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:41.988+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:42.057 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.057 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:42.108 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:42.105+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:42.193 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:42.190+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:42.211 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.211 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.220 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:42.293 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:42.291+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:42.323 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.324 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:42.394 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:42.392+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:42.476 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.476 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.484 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:42.492 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 6, 7, 5, 2] 2024-09-17T22:59:42.492 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:59:42.492 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T22:59:42.492 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:59:42.529 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:42.526+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:42.590 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.590 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.595 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:42.593+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:42.600 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:42.696 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:42.693+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:42.744 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.745 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.753 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:42.797 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:42.794+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "epoch": 155, 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "fsid": "33081d94-d15e-4bd4-ba4c-30022eb1be7a", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "created": "2024-09-17T22:56:50.056303+0000", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "modified": "2024-09-17T22:59:41.939065+0000", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "last_up_change": "2024-09-17T22:59:09.941537+0000", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "last_in_change": "2024-09-17T22:58:51.164342+0000", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_num": 5799936, 2024-09-17T22:59:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_set": [ 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "pglog_hardlimit", 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "purged_snapdirs", 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_deletes", 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "sortbitwise" 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_version": 7, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_snapshot": "", 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_max": 3, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd": 8, 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:59:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "min_compat_client": "luminous", 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "require_osd_release": "squid", 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_crimson": false, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pools": [ 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 1, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": ".mgr", 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:02.221581+0000", 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool", 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 1, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 1, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 1, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 1, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 1, 2024-09-17T22:59:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "17", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "options": { 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_max": 32, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_min": 1 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": {} 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 8, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 8, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.25, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 2, 2024-09-17T22:59:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "rbd", 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:08.859324+0000", 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 8, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 8, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 8, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 8, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 8, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "21", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 2, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 21, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:59:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 2.9900000095367432, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 2.9900000095367432, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 2.630000114440918, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 2.630000114440918, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": "pool_client.0", 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "create_time": "2024-09-17T22:57:17.992936+0000", 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 8193, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "type": 1, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 2, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": 1, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_rule": 0, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:59:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "is_stretch_pool": false, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "object_hash": 2, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": 32, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num": 32, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_placement_num_target": 32, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_target": 32, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num_pending": 32, 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "last_pg_merge_meta": { 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "source_pgid": "0.0", 2024-09-17T22:59:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "ready_epoch": 0, 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_started": 0, 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_epoch_clean": 0, 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "source_version": "0'0", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "target_version": "0'0" 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_change": "155", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend": "0", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "auid": 0, 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:59:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_seq": 54, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_epoch": 155, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_snaps": [], 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps": "[]", 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_bytes": 0, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "quota_max_objects": 0, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "tiers": [], 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "tier_of": -1, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "read_tier": -1, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "write_tier": -1, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_mode": "none", 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_bytes": 0, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "target_max_objects": 0, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:59:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_flush_age": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "cache_min_evict_age": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profile": "", 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_params": { 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "none" 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_period": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_count": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "use_gmt_hitset": true, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:59:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "grade_table": [], 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "stripe_width": 0, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "expected_num_objects": 0, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "fast_read": false, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "options": {}, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "application_metadata": { 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd": {} 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "read_balance": { 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "score_type": "Fair distribution", 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "score_acting": 1.75, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "score_stable": 1.75, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "optimal_score": 1, 2024-09-17T22:59:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_acting": 1.75, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "raw_score_stable": 1.75, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity_weighted": 0.89504432678222656, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "892321c3-409f-40e6-bfe1-c87f464d030f", 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6808", 2024-09-17T22:59:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6809", 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6810", 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6811", 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6814", 2024-09-17T22:59:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6815", 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.840 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6812", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6813", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2087020665 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6809/2087020665", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6811/2087020665", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6815/2087020665", 2024-09-17T22:59:42.841 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6813/2087020665", 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "cd777f15-a6ad-4f0c-820b-ef529dd9f008", 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.842 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6800", 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.843 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6801", 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.844 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6802", 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6803", 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.845 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6806", 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6807", 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.846 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6804", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6805", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 237398697 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6801/237398697", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6803/237398697", 2024-09-17T22:59:42.847 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6807/237398697", 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6805/237398697", 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "bd68e905-6e66-4735-9de8-66a99530420f", 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.848 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.849 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 14, 2024-09-17T22:59:42.849 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 74, 2024-09-17T22:59:42.849 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 123, 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 130, 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 75, 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6816", 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6817", 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.853 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6818", 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6819", 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6822", 2024-09-17T22:59:42.854 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6823", 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.855 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6820", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6821", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 3362624831 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6817/3362624831", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6819/3362624831", 2024-09-17T22:59:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6823/3362624831", 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6821/3362624831", 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "28b3b7ca-d7d7-4ed9-8dfe-113d3579209c", 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.857 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 14, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6824", 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6825", 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6826", 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.859 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6827", 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6830", 2024-09-17T22:59:42.860 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6831", 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.861 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6828", 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6829", 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2987856795 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.862 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.159:6825/2987856795", 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.159:6827/2987856795", 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.159:6831/2987856795", 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.159:6829/2987856795", 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.863 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "d379c252-e66a-4d0e-af0a-81f6bda48c08", 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:59:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 135, 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6826", 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6827", 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6828", 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6829", 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6832", 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6833", 2024-09-17T22:59:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6830", 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6831", 2024-09-17T22:59:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2123531187 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6827/2123531187", 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6829/2123531187", 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6833/2123531187", 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6831/2123531187", 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "e78a100f-1ff3-4af6-abd8-a069030ec951", 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 15, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 45, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 71, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 126, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 46, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6818", 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6819", 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6820", 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6821", 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6824", 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6825", 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6822", 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6823", 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 587621150 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6819/587621150", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6821/587621150", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6825/587621150", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6823/587621150", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "fc1af848-dfaa-4e40-a0af-8982c70f7ea9", 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 1, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 15, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 134, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6810", 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6811", 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6812", 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6813", 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6816", 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6817", 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6814", 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.879 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6815", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 1505991260 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6811/1505991260", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6813/1505991260", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6817/1505991260", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6815/1505991260", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.880 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "uuid": "ccd75a18-ded4-4f75-a09a-926da5362a7f", 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "up": 1, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "in": 1, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "weight": 1, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_affinity": 0.1603546142578125, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_begin": 0, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "last_clean_end": 0, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "up_from": 13, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "up_thru": 75, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "down_at": 0, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "lost_at": 0, 2024-09-17T22:59:42.881 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addrs": { 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6800", 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6801", 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addrs": { 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6802", 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6803", 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.883 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addrs": { 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6806", 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6807", 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.884 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addrs": { 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6804", 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.171:6805", 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 585405482 2024-09-17T22:59:42.885 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "public_addr": "172.21.15.171:6801/585405482", 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_addr": "172.21.15.171:6803/585405482", 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_back_addr": "172.21.15.171:6807/585405482", 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "heartbeat_front_addr": "172.21.15.171:6805/585405482", 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "state": [ 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "exists", 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: "up" 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.886 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_xinfo": [ 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 0, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.101775+0000", 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 1, 2024-09-17T22:59:42.887 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.929708+0000", 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 2, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.888 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.562720+0000", 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 75 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 3, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.889 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.569375+0000", 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 4, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:57:00.002784+0000", 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.890 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 5, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.775667+0000", 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 46 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 6, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.891 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:59.558418+0000", 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": 7, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "down_stamp": "0.000000", 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_probability": 0, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "laggy_interval": 0, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "features": 4540701547738038271, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "old_weight": 0, 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "last_purged_snaps_scrub": "2024-09-17T22:56:58.840790+0000", 2024-09-17T22:59:42.892 INFO:teuthology.orchestra.run.smithi159.stdout: "dead_epoch": 0 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap": [ 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: "pgid": "3.9", 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: 6, 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: 3 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: "pgid": "3.11", 2024-09-17T22:59:42.893 INFO:teuthology.orchestra.run.smithi159.stdout: "osds": [ 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: 5, 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: 2 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_items": [], 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_upmap_primaries": [], 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_temp": [], 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "primary_temp": [], 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "blocklist": {}, 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "range_blocklist": {}, 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "erasure_code_profiles": { 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "default": { 2024-09-17T22:59:42.894 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-failure-domain": "osd", 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "k": "2", 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "m": "1", 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "plugin": "jerasure", 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "technique": "reed_sol_van" 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "removed_snaps_queue": [ 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 9, 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 9 2024-09-17T22:59:42.895 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: "new_removed_snaps": [ 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 3, 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: "snaps": [ 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: "begin": 9, 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: "length": 1 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.896 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "new_purged_snaps": [], 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_node_flags": {}, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "device_class_flags": {}, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode": { 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_enabled": false, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_bucket_count": 0, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: "stretch_mode_bucket": 0 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:42.897 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:42.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:42.900 INFO:tasks.thrashosds.thrasher:j is {'epoch': 155, 'fsid': '33081d94-d15e-4bd4-ba4c-30022eb1be7a', 'created': '2024-09-17T22:56:50.056303+0000', 'modified': '2024-09-17T22:59:41.939065+0000', 'last_up_change': '2024-09-17T22:59:09.941537+0000', 'last_in_change': '2024-09-17T22:58:51.164342+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:57:02.221581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:57:08.859324+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:57:17.992936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 155, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8950443267822266, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': '892321c3-409f-40e6-bfe1-c87f464d030f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6808', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6809', 'nonce': 2087020665}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6810', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6811', 'nonce': 2087020665}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6814', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6815', 'nonce': 2087020665}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6812', 'nonce': 2087020665}, {'type': 'v1', 'addr': '172.21.15.159:6813', 'nonce': 2087020665}]}, 'public_addr': '172.21.15.159:6809/2087020665', 'cluster_addr': '172.21.15.159:6811/2087020665', 'heartbeat_back_addr': '172.21.15.159:6815/2087020665', 'heartbeat_front_addr': '172.21.15.159:6813/2087020665', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'cd777f15-a6ad-4f0c-820b-ef529dd9f008', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6800', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6801', 'nonce': 237398697}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6802', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6803', 'nonce': 237398697}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6806', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6807', 'nonce': 237398697}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6804', 'nonce': 237398697}, {'type': 'v1', 'addr': '172.21.15.159:6805', 'nonce': 237398697}]}, 'public_addr': '172.21.15.159:6801/237398697', 'cluster_addr': '172.21.15.159:6803/237398697', 'heartbeat_back_addr': '172.21.15.159:6807/237398697', 'heartbeat_front_addr': '172.21.15.159:6805/237398697', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bd68e905-6e66-4735-9de8-66a99530420f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 74, 'up_from': 123, 'up_thru': 130, 'down_at': 75, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6816', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6817', 'nonce': 3362624831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6818', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6819', 'nonce': 3362624831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6822', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6823', 'nonce': 3362624831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6820', 'nonce': 3362624831}, {'type': 'v1', 'addr': '172.21.15.159:6821', 'nonce': 3362624831}]}, 'public_addr': '172.21.15.159:6817/3362624831', 'cluster_addr': '172.21.15.159:6819/3362624831', 'heartbeat_back_addr': '172.21.15.159:6823/3362624831', 'heartbeat_front_addr': '172.21.15.159:6821/3362624831', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '28b3b7ca-d7d7-4ed9-8dfe-113d3579209c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6824', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6825', 'nonce': 2987856795}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6826', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6827', 'nonce': 2987856795}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6830', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6831', 'nonce': 2987856795}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6828', 'nonce': 2987856795}, {'type': 'v1', 'addr': '172.21.15.159:6829', 'nonce': 2987856795}]}, 'public_addr': '172.21.15.159:6825/2987856795', 'cluster_addr': '172.21.15.159:6827/2987856795', 'heartbeat_back_addr': '172.21.15.159:6831/2987856795', 'heartbeat_front_addr': '172.21.15.159:6829/2987856795', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd379c252-e66a-4d0e-af0a-81f6bda48c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6826', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6827', 'nonce': 2123531187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6828', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6829', 'nonce': 2123531187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2123531187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6830', 'nonce': 2123531187}, {'type': 'v1', 'addr': '172.21.15.171:6831', 'nonce': 2123531187}]}, 'public_addr': '172.21.15.171:6827/2123531187', 'cluster_addr': '172.21.15.171:6829/2123531187', 'heartbeat_back_addr': '172.21.15.171:6833/2123531187', 'heartbeat_front_addr': '172.21.15.171:6831/2123531187', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e78a100f-1ff3-4af6-abd8-a069030ec951', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 45, 'up_from': 71, 'up_thru': 126, 'down_at': 46, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6818', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6819', 'nonce': 587621150}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6820', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6821', 'nonce': 587621150}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6824', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6825', 'nonce': 587621150}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6822', 'nonce': 587621150}, {'type': 'v1', 'addr': '172.21.15.171:6823', 'nonce': 587621150}]}, 'public_addr': '172.21.15.171:6819/587621150', 'cluster_addr': '172.21.15.171:6821/587621150', 'heartbeat_back_addr': '172.21.15.171:6825/587621150', 'heartbeat_front_addr': '172.21.15.171:6823/587621150', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'fc1af848-dfaa-4e40-a0af-8982c70f7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6810', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6811', 'nonce': 1505991260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6812', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6813', 'nonce': 1505991260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6816', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6817', 'nonce': 1505991260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6814', 'nonce': 1505991260}, {'type': 'v1', 'addr': '172.21.15.171:6815', 'nonce': 1505991260}]}, 'public_addr': '172.21.15.171:6811/1505991260', 'cluster_addr': '172.21.15.171:6813/1505991260', 'heartbeat_back_addr': '172.21.15.171:6817/1505991260', 'heartbeat_front_addr': '172.21.15.171:6815/1505991260', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ccd75a18-ded4-4f75-a09a-926da5362a7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1603546142578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6800', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6801', 'nonce': 585405482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6802', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6803', 'nonce': 585405482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6806', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6807', 'nonce': 585405482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6804', 'nonce': 585405482}, {'type': 'v1', 'addr': '172.21.15.171:6805', 'nonce': 585405482}]}, 'public_addr': '172.21.15.171:6801/585405482', 'cluster_addr': '172.21.15.171:6803/585405482', 'heartbeat_back_addr': '172.21.15.171:6807/585405482', 'heartbeat_front_addr': '172.21.15.171:6805/585405482', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.101775+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.929708+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.562720+0000', 'dead_epoch': 75}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.569375+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:57:00.002784+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.775667+0000', 'dead_epoch': 46}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:59.558418+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:56:58.840790+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.9', 'osds': [6, 3]}, {'pgid': '3.11', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:59:42.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:59:42.900 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:42.900 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:42.901 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:42.897+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:42.905 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:42.955 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is set 2024-09-17T22:59:42.999 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:42.997+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:43.049 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:43.049 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:43.070 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:43.070 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:43.076 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:43.076 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:43.086 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:43.090 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:43.090 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:43.100 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:43.098+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:43.107 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:43.107 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:43.124 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:43.141 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:43.141 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:43.158 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:43.158 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:43.175 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:43.195 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:43.195 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:43.195 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:43.195 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:43.195 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:43.200 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:43.198+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:43.204 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:43.253 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T22:59:43.254 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T22:59:43.266 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T22:59:42.954030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326211932,"num_objects":618,"num_object_clones":58,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":618,"num_whiteouts":490,"num_read":260658,"num_read_kb":795171,"num_write":23308,"num_write_kb":368657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20696,"ondisk_log_size":20696,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":445000,"kb_used_data":59008,"kb_used_omap":41,"kb_used_meta":385750,"kb_avail":754529720,"statfs":{"total":773094113280,"available":772638433280,"internally_reserved":0,"allocated":60424192,"data_stored":57697249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":395008281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-123907256,"num_objects":-145,"num_object_clones":-51,"num_object_copies":-290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-145,"num_whiteouts":-44,"num_read":1432,"num_read_kb":1215,"num_write":198,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019355"},"pg_stats":[{"pgid":"3.1f","version":"155'461","reported_seq":4746,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.954624+0000","last_change":"2024-09-17T22:59:41.954964+0000","last_active":"2024-09-17T22:59:41.954624+0000","last_peered":"2024-09-17T22:59:41.954624+0000","last_clean":"2024-09-17T22:59:41.954624+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T22:59:41.954624+0000","last_undegraded":"2024-09-17T22:59:41.954624+0000","last_fullsized":"2024-09-17T22:59:41.954624+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:10:13.465438+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.011650749,"stat_sum":{"num_bytes":6153806,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":4250,"num_read_kb":23847,"num_write":445,"num_write_kb":24358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.1e","version":"155'321","reported_seq":3417,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.960806+0000","last_change":"2024-09-17T22:59:41.960806+0000","last_active":"2024-09-17T22:59:41.960806+0000","last_peered":"2024-09-17T22:59:41.960806+0000","last_clean":"2024-09-17T22:59:41.960806+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T22:59:41.960806+0000","last_undegraded":"2024-09-17T22:59:41.960806+0000","last_fullsized":"2024-09-17T22:59:41.960806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.647884+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.012103648999999999,"stat_sum":{"num_bytes":8575710,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":13,"num_read":2653,"num_read_kb":17544,"num_write":282,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.1d","version":"153'572","reported_seq":9953,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.949155+0000","last_change":"2024-09-17T22:59:40.949689+0000","last_active":"2024-09-17T22:59:40.949155+0000","last_peered":"2024-09-17T22:59:40.949155+0000","last_clean":"2024-09-17T22:59:40.949155+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T22:59:40.949155+0000","last_undegraded":"2024-09-17T22:59:40.949155+0000","last_fullsized":"2024-09-17T22:59:40.949155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:31:14.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377489,"stat_sum":{"num_bytes":2767678,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":17,"num_read":10459,"num_read_kb":31556,"num_write":583,"num_write_kb":6661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.1c","version":"154'628","reported_seq":6131,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.036992+0000","last_change":"2024-09-17T22:59:40.950843+0000","last_active":"2024-09-17T22:59:41.036992+0000","last_peered":"2024-09-17T22:59:41.036992+0000","last_clean":"2024-09-17T22:59:41.036992+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T22:59:41.036992+0000","last_undegraded":"2024-09-17T22:59:41.036992+0000","last_fullsized":"2024-09-17T22:59:41.036992+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:25:19.310934+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047785070000000004,"stat_sum":{"num_bytes":1404588,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":13,"num_read":6848,"num_read_kb":17381,"num_write":653,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.1b","version":"155'753","reported_seq":15183,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.962088+0000","last_change":"2024-09-17T22:59:41.962088+0000","last_active":"2024-09-17T22:59:41.962088+0000","last_peered":"2024-09-17T22:59:41.962088+0000","last_clean":"2024-09-17T22:59:41.962088+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T22:59:41.962088+0000","last_undegraded":"2024-09-17T22:59:41.962088+0000","last_fullsized":"2024-09-17T22:59:41.962088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:53:33.406882+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.018146224999999998,"stat_sum":{"num_bytes":11217391,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":757,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.1a","version":"150'412","reported_seq":10461,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.063723+0000","last_change":"2024-09-17T22:59:37.930203+0000","last_active":"2024-09-17T22:59:38.063723+0000","last_peered":"2024-09-17T22:59:38.063723+0000","last_clean":"2024-09-17T22:59:38.063723+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T22:59:38.063723+0000","last_undegraded":"2024-09-17T22:59:38.063723+0000","last_fullsized":"2024-09-17T22:59:38.063723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:09.408180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040421199999999998,"stat_sum":{"num_bytes":14405655,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":10732,"num_read_kb":23422,"num_write":399,"num_write_kb":11995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.19","version":"155'779","reported_seq":5632,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.949190+0000","last_change":"2024-09-17T22:59:41.949190+0000","last_active":"2024-09-17T22:59:41.949190+0000","last_peered":"2024-09-17T22:59:41.949190+0000","last_clean":"2024-09-17T22:59:41.949190+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T22:59:41.949190+0000","last_undegraded":"2024-09-17T22:59:41.949190+0000","last_fullsized":"2024-09-17T22:59:41.949190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:56.194884+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062404310000000003,"stat_sum":{"num_bytes":131215,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":16,"num_read":6850,"num_read_kb":19732,"num_write":870,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.18","version":"155'503","reported_seq":5885,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.962847+0000","last_change":"2024-09-17T22:59:41.963155+0000","last_active":"2024-09-17T22:59:41.962847+0000","last_peered":"2024-09-17T22:59:41.962847+0000","last_clean":"2024-09-17T22:59:41.962847+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T22:59:41.962847+0000","last_undegraded":"2024-09-17T22:59:41.962847+0000","last_fullsized":"2024-09-17T22:59:41.962847+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:33.787931+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014084199,"stat_sum":{"num_bytes":180093,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":4744,"num_read_kb":27535,"num_write":465,"num_write_kb":27831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.17","version":"153'687","reported_seq":4696,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.959503+0000","last_change":"2024-09-17T22:59:41.959956+0000","last_active":"2024-09-17T22:59:41.959503+0000","last_peered":"2024-09-17T22:59:41.959503+0000","last_clean":"2024-09-17T22:59:41.959503+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T22:59:41.959503+0000","last_undegraded":"2024-09-17T22:59:41.959503+0000","last_fullsized":"2024-09-17T22:59:41.959503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:01.398148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015447361,"stat_sum":{"num_bytes":12780562,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":5593,"num_read_kb":21131,"num_write":818,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.16","version":"147'1325","reported_seq":9701,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.947511+0000","last_change":"2024-09-17T22:59:40.948534+0000","last_active":"2024-09-17T22:59:40.947511+0000","last_peered":"2024-09-17T22:59:40.947511+0000","last_clean":"2024-09-17T22:59:40.947511+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T22:59:40.947511+0000","last_undegraded":"2024-09-17T22:59:40.947511+0000","last_fullsized":"2024-09-17T22:59:40.947511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:27.393967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024689249999999999,"stat_sum":{"num_bytes":21652087,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":13659,"num_read_kb":44273,"num_write":1907,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.15","version":"155'481","reported_seq":4973,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.964416+0000","last_change":"2024-09-17T22:59:41.964416+0000","last_active":"2024-09-17T22:59:41.964416+0000","last_peered":"2024-09-17T22:59:41.964416+0000","last_clean":"2024-09-17T22:59:41.964416+0000","last_became_active":"2024-09-17T22:58:46.132716+0000","last_became_peered":"2024-09-17T22:58:46.132716+0000","last_unstale":"2024-09-17T22:59:41.964416+0000","last_undegraded":"2024-09-17T22:59:41.964416+0000","last_fullsized":"2024-09-17T22:59:41.964416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:42.700264+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015286190999999999,"stat_sum":{"num_bytes":41278932,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":438,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.14","version":"151'489","reported_seq":2920,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:37.930222+0000","last_change":"2024-09-17T22:59:37.930222+0000","last_active":"2024-09-17T22:59:37.930222+0000","last_peered":"2024-09-17T22:59:37.930222+0000","last_clean":"2024-09-17T22:59:37.930222+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T22:59:37.930222+0000","last_undegraded":"2024-09-17T22:59:37.930222+0000","last_fullsized":"2024-09-17T22:59:37.930222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:24.358528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033864300000000002,"stat_sum":{"num_bytes":4194357,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":22,"num_read":3529,"num_read_kb":11456,"num_write":471,"num_write_kb":5679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":293,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.004135+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T22:59:38.004135+0000","last_peered":"2024-09-17T22:59:38.004135+0000","last_clean":"2024-09-17T22:59:38.004135+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T22:59:38.004135+0000","last_undegraded":"2024-09-17T22:59:38.004135+0000","last_fullsized":"2024-09-17T22:59:38.004135+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"155'667","reported_seq":10323,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.959478+0000","last_change":"2024-09-17T22:59:41.959478+0000","last_active":"2024-09-17T22:59:41.959478+0000","last_peered":"2024-09-17T22:59:41.959478+0000","last_clean":"2024-09-17T22:59:41.959478+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T22:59:41.959478+0000","last_undegraded":"2024-09-17T22:59:41.959478+0000","last_fullsized":"2024-09-17T22:59:41.959478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:38.150058+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015162337999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":12268,"num_read_kb":26380,"num_write":552,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":288,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.943218+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T22:59:41.943218+0000","last_peered":"2024-09-17T22:59:41.943218+0000","last_clean":"2024-09-17T22:59:41.943218+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T22:59:41.943218+0000","last_undegraded":"2024-09-17T22:59:41.943218+0000","last_fullsized":"2024-09-17T22:59:41.943218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"155'509","reported_seq":9745,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.960259+0000","last_change":"2024-09-17T22:59:41.960594+0000","last_active":"2024-09-17T22:59:41.960259+0000","last_peered":"2024-09-17T22:59:41.960259+0000","last_clean":"2024-09-17T22:59:41.960259+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T22:59:41.960259+0000","last_undegraded":"2024-09-17T22:59:41.960259+0000","last_fullsized":"2024-09-17T22:59:41.960259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:00.801839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011643161000000001,"stat_sum":{"num_bytes":249822,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":18,"num_read":9662,"num_read_kb":29730,"num_write":508,"num_write_kb":5475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.948481+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T22:59:40.948481+0000","last_peered":"2024-09-17T22:59:40.948481+0000","last_clean":"2024-09-17T22:59:40.948481+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T22:59:40.948481+0000","last_undegraded":"2024-09-17T22:59:40.948481+0000","last_fullsized":"2024-09-17T22:59:40.948481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10020,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.957656+0000","last_change":"2024-09-17T22:59:41.957656+0000","last_active":"2024-09-17T22:59:41.957656+0000","last_peered":"2024-09-17T22:59:41.957656+0000","last_clean":"2024-09-17T22:59:41.957656+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T22:59:41.957656+0000","last_undegraded":"2024-09-17T22:59:41.957656+0000","last_fullsized":"2024-09-17T22:59:41.957656+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:58:53.140010+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.014391536999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":288,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.943261+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T22:59:41.943261+0000","last_peered":"2024-09-17T22:59:41.943261+0000","last_clean":"2024-09-17T22:59:41.943261+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T22:59:41.943261+0000","last_undegraded":"2024-09-17T22:59:41.943261+0000","last_fullsized":"2024-09-17T22:59:41.943261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"153'752","reported_seq":4076,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.042807+0000","last_change":"2024-09-17T22:59:40.948312+0000","last_active":"2024-09-17T22:59:41.042807+0000","last_peered":"2024-09-17T22:59:41.042807+0000","last_clean":"2024-09-17T22:59:41.042807+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T22:59:41.042807+0000","last_undegraded":"2024-09-17T22:59:41.042807+0000","last_fullsized":"2024-09-17T22:59:41.042807+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:21.911389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021969619999999998,"stat_sum":{"num_bytes":5861094,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":5850,"num_read_kb":19415,"num_write":915,"num_write_kb":12178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.3","version":"151'707","reported_seq":5322,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.019339+0000","last_change":"2024-09-17T22:59:37.930519+0000","last_active":"2024-09-17T22:59:38.019339+0000","last_peered":"2024-09-17T22:59:38.019339+0000","last_clean":"2024-09-17T22:59:38.019339+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T22:59:38.019339+0000","last_undegraded":"2024-09-17T22:59:38.019339+0000","last_fullsized":"2024-09-17T22:59:38.019339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:39:53.463763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059625900000000003,"stat_sum":{"num_bytes":17228159,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":5386,"num_read_kb":23982,"num_write":692,"num_write_kb":9747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"2.2","version":"21'2","reported_seq":112,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.945941+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T22:59:40.945941+0000","last_peered":"2024-09-17T22:59:40.945941+0000","last_clean":"2024-09-17T22:59:40.945941+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T22:59:40.945941+0000","last_undegraded":"2024-09-17T22:59:40.945941+0000","last_fullsized":"2024-09-17T22:59:40.945941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'387","reported_seq":19128,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.947062+0000","last_change":"2024-09-17T22:59:41.948406+0000","last_active":"2024-09-17T22:59:41.947062+0000","last_peered":"2024-09-17T22:59:41.947062+0000","last_clean":"2024-09-17T22:59:41.947062+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T22:59:41.947062+0000","last_undegraded":"2024-09-17T22:59:41.947062+0000","last_fullsized":"2024-09-17T22:59:41.947062+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:05.732601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0053815829999999997,"stat_sum":{"num_bytes":16,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":19390,"num_read_kb":25737,"num_write":336,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":174,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.945327+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T22:59:38.945327+0000","last_peered":"2024-09-17T22:59:38.945327+0000","last_clean":"2024-09-17T22:59:38.945327+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T22:59:38.945327+0000","last_undegraded":"2024-09-17T22:59:38.945327+0000","last_fullsized":"2024-09-17T22:59:38.945327+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"153'1260","reported_seq":13273,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.943467+0000","last_change":"2024-09-17T22:59:41.944063+0000","last_active":"2024-09-17T22:59:41.943467+0000","last_peered":"2024-09-17T22:59:41.943467+0000","last_clean":"2024-09-17T22:59:41.943467+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T22:59:41.943467+0000","last_undegraded":"2024-09-17T22:59:41.943467+0000","last_fullsized":"2024-09-17T22:59:41.943467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1260,"log_dups_size":0,"ondisk_log_size":1260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:42:12.597199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001202222,"stat_sum":{"num_bytes":393216,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":16,"num_read":16206,"num_read_kb":43141,"num_write":1908,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.943000+0000","last_change":"2024-09-17T22:58:46.131382+0000","last_active":"2024-09-17T22:59:41.943000+0000","last_peered":"2024-09-17T22:59:41.943000+0000","last_clean":"2024-09-17T22:59:41.943000+0000","last_became_active":"2024-09-17T22:58:46.130955+0000","last_became_peered":"2024-09-17T22:58:46.130955+0000","last_unstale":"2024-09-17T22:59:41.943000+0000","last_undegraded":"2024-09-17T22:59:41.943000+0000","last_fullsized":"2024-09-17T22:59:41.943000+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:57.813197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"155'857","reported_seq":6245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.964870+0000","last_change":"2024-09-17T22:59:41.965169+0000","last_active":"2024-09-17T22:59:41.964870+0000","last_peered":"2024-09-17T22:59:41.964870+0000","last_clean":"2024-09-17T22:59:41.964870+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T22:59:41.964870+0000","last_undegraded":"2024-09-17T22:59:41.964870+0000","last_fullsized":"2024-09-17T22:59:41.964870+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:02.770137+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015881556000000002,"stat_sum":{"num_bytes":6205001,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":7030,"num_read_kb":16025,"num_write":1086,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"2.3","version":"19'1","reported_seq":115,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.945969+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T22:59:40.945969+0000","last_peered":"2024-09-17T22:59:40.945969+0000","last_clean":"2024-09-17T22:59:40.945969+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T22:59:40.945969+0000","last_undegraded":"2024-09-17T22:59:40.945969+0000","last_fullsized":"2024-09-17T22:59:40.945969+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":282,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.943138+0000","last_change":"2024-09-17T22:58:46.130584+0000","last_active":"2024-09-17T22:59:41.943138+0000","last_peered":"2024-09-17T22:59:41.943138+0000","last_clean":"2024-09-17T22:59:41.943138+0000","last_became_active":"2024-09-17T22:58:46.130202+0000","last_became_peered":"2024-09-17T22:58:46.130202+0000","last_unstale":"2024-09-17T22:59:41.943138+0000","last_undegraded":"2024-09-17T22:59:41.943138+0000","last_fullsized":"2024-09-17T22:59:41.943138+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:32.916711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"153'812","reported_seq":3922,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.949389+0000","last_change":"2024-09-17T22:59:41.953618+0000","last_active":"2024-09-17T22:59:41.949389+0000","last_peered":"2024-09-17T22:59:41.949389+0000","last_clean":"2024-09-17T22:59:41.949389+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T22:59:41.949389+0000","last_undegraded":"2024-09-17T22:59:41.949389+0000","last_fullsized":"2024-09-17T22:59:41.949389+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:35.632301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004719455,"stat_sum":{"num_bytes":29927651,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":5378,"num_read_kb":13694,"num_write":1117,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.9","version":"151'548","reported_seq":4227,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.005807+0000","last_change":"2024-09-17T22:59:38.005807+0000","last_active":"2024-09-17T22:59:38.005807+0000","last_peered":"2024-09-17T22:59:38.005807+0000","last_clean":"2024-09-17T22:59:38.005807+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T22:59:38.005807+0000","last_undegraded":"2024-09-17T22:59:38.005807+0000","last_fullsized":"2024-09-17T22:59:38.005807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:29:14.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014585240000000001,"stat_sum":{"num_bytes":3976958,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":5472,"num_read_kb":14550,"num_write":527,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.a","version":"151'552","reported_seq":3571,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.009084+0000","last_change":"2024-09-17T22:59:38.004759+0000","last_active":"2024-09-17T22:59:38.009084+0000","last_peered":"2024-09-17T22:59:38.009084+0000","last_clean":"2024-09-17T22:59:38.009084+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T22:59:38.009084+0000","last_undegraded":"2024-09-17T22:59:38.009084+0000","last_fullsized":"2024-09-17T22:59:38.009084+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:30.664700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046043899999999999,"stat_sum":{"num_bytes":2312117,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":17,"num_read":3616,"num_read_kb":19051,"num_write":633,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.b","version":"155'557","reported_seq":6446,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.956635+0000","last_change":"2024-09-17T22:59:41.956923+0000","last_active":"2024-09-17T22:59:41.956635+0000","last_peered":"2024-09-17T22:59:41.956635+0000","last_clean":"2024-09-17T22:59:41.956635+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T22:59:41.956635+0000","last_undegraded":"2024-09-17T22:59:41.956635+0000","last_fullsized":"2024-09-17T22:59:41.956635+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:34.162620+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079782099999999995,"stat_sum":{"num_bytes":5214241,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":16,"num_read":6521,"num_read_kb":16591,"num_write":535,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.c","version":"153'418","reported_seq":5171,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.946153+0000","last_change":"2024-09-17T22:59:40.947463+0000","last_active":"2024-09-17T22:59:40.946153+0000","last_peered":"2024-09-17T22:59:40.946153+0000","last_clean":"2024-09-17T22:59:40.946153+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T22:59:40.946153+0000","last_undegraded":"2024-09-17T22:59:40.946153+0000","last_fullsized":"2024-09-17T22:59:40.946153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:19.355386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014557179999999999,"stat_sum":{"num_bytes":12494754,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":5440,"num_read_kb":18887,"num_write":479,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.d","version":"154'446","reported_seq":5086,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.020991+0000","last_change":"2024-09-17T22:59:40.949073+0000","last_active":"2024-09-17T22:59:41.020991+0000","last_peered":"2024-09-17T22:59:41.020991+0000","last_clean":"2024-09-17T22:59:41.020991+0000","last_became_active":"2024-09-17T22:58:54.861414+0000","last_became_peered":"2024-09-17T22:58:54.861414+0000","last_unstale":"2024-09-17T22:59:41.020991+0000","last_undegraded":"2024-09-17T22:59:41.020991+0000","last_fullsized":"2024-09-17T22:59:41.020991+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:33:50.595806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002849776,"stat_sum":{"num_bytes":12729433,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":440,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.e","version":"145'345","reported_seq":2124,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:37.930513+0000","last_change":"2024-09-17T22:59:37.930513+0000","last_active":"2024-09-17T22:59:37.930513+0000","last_peered":"2024-09-17T22:59:37.930513+0000","last_clean":"2024-09-17T22:59:37.930513+0000","last_became_active":"2024-09-17T22:59:22.867399+0000","last_became_peered":"2024-09-17T22:59:22.867399+0000","last_unstale":"2024-09-17T22:59:37.930513+0000","last_undegraded":"2024-09-17T22:59:37.930513+0000","last_fullsized":"2024-09-17T22:59:37.930513+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:27.524754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061193200000000004,"stat_sum":{"num_bytes":4032164,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":20,"num_read":1602,"num_read_kb":7801,"num_write":332,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.f","version":"151'1090","reported_seq":9263,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.079987+0000","last_change":"2024-09-17T22:59:37.930763+0000","last_active":"2024-09-17T22:59:38.079987+0000","last_peered":"2024-09-17T22:59:38.079987+0000","last_clean":"2024-09-17T22:59:38.079987+0000","last_became_active":"2024-09-17T22:58:46.132674+0000","last_became_peered":"2024-09-17T22:58:46.132674+0000","last_unstale":"2024-09-17T22:59:38.079987+0000","last_undegraded":"2024-09-17T22:59:38.079987+0000","last_fullsized":"2024-09-17T22:59:38.079987+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:56:12.947612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083814700000000002,"stat_sum":{"num_bytes":4456464,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":12331,"num_read_kb":34886,"num_write":1391,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.10","version":"150'687","reported_seq":5717,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T22:59:38.004287+0000","last_change":"2024-09-17T22:59:38.004673+0000","last_active":"2024-09-17T22:59:38.004287+0000","last_peered":"2024-09-17T22:59:38.004287+0000","last_clean":"2024-09-17T22:59:38.004287+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T22:59:38.004287+0000","last_undegraded":"2024-09-17T22:59:38.004287+0000","last_fullsized":"2024-09-17T22:59:38.004287+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:51:14.946011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053967000000000002,"stat_sum":{"num_bytes":52050061,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":5722,"num_read_kb":33563,"num_write":875,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"d","length":"b"}]},{"pgid":"3.11","version":"154'514","reported_seq":4585,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:41.065104+0000","last_change":"2024-09-17T22:59:40.947122+0000","last_active":"2024-09-17T22:59:41.065104+0000","last_peered":"2024-09-17T22:59:41.065104+0000","last_clean":"2024-09-17T22:59:41.065104+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T22:59:41.065104+0000","last_undegraded":"2024-09-17T22:59:41.065104+0000","last_fullsized":"2024-09-17T22:59:41.065104+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:43.725441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011848060000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4672,"num_read_kb":17212,"num_write":484,"num_write_kb":5650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.12","version":"155'804","reported_seq":13497,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:59:42.040128+0000","last_change":"2024-09-17T22:59:41.951930+0000","last_active":"2024-09-17T22:59:42.040128+0000","last_peered":"2024-09-17T22:59:42.040128+0000","last_clean":"2024-09-17T22:59:42.040128+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T22:59:42.040128+0000","last_undegraded":"2024-09-17T22:59:42.040128+0000","last_fullsized":"2024-09-17T22:59:42.040128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:52:30.544146+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0092379599999999999,"stat_sum":{"num_bytes":19922944,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":991,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"9"}]},{"pgid":"3.13","version":"153'657","reported_seq":9146,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:59:40.948786+0000","last_change":"2024-09-17T22:59:40.948786+0000","last_active":"2024-09-17T22:59:40.948786+0000","last_peered":"2024-09-17T22:59:40.948786+0000","last_clean":"2024-09-17T22:59:40.948786+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T22:59:40.948786+0000","last_undegraded":"2024-09-17T22:59:40.948786+0000","last_fullsized":"2024-09-17T22:59:40.948786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:51.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053722700000000004,"stat_sum":{"num_bytes":23825376,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":23,"num_read":10577,"num_read_kb":26820,"num_write":713,"num_write_kb":10350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":325621545,"num_objects":614,"num_object_clones":58,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":614,"num_whiteouts":490,"num_read":260542,"num_read_kb":795071,"num_write":23147,"num_write_kb":366239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34045952,"data_stored":37760201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":20576,"ondisk_log_size":20576,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":7416,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":7593984,"data_stored":7197835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678036,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45916,"kb_used_data":4616,"kb_used_omap":17,"kb_used_meta":41262,"kb_avail":94325924,"statfs":{"total":96636764160,"available":96589746176,"internally_reserved":0,"allocated":4726784,"data_stored":4122622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":42252486},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61400,"kb_used_data":16320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310440,"statfs":{"total":96636764160,"available":96573890560,"internally_reserved":0,"allocated":16711680,"data_stored":17468022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64564,"kb_used_data":7012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307276,"statfs":{"total":96636764160,"available":96570650624,"internally_reserved":0,"allocated":7180288,"data_stored":6738909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977415,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30272,"kb_used_data":1832,"kb_used_omap":14,"kb_used_meta":28401,"kb_avail":94341568,"statfs":{"total":96636764160,"available":96605765632,"internally_reserved":0,"allocated":1875968,"data_stored":941989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":29083050},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55684,"kb_used_data":6444,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94316156,"statfs":{"total":96636764160,"available":96579743744,"internally_reserved":0,"allocated":6598656,"data_stored":6152584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67076,"kb_used_data":3936,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94304764,"statfs":{"total":96636764160,"available":96568078336,"internally_reserved":0,"allocated":4030464,"data_stored":3042480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68996,"kb_used_data":11432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94302844,"statfs":{"total":96636764160,"available":96566112256,"internally_reserved":0,"allocated":11706368,"data_stored":12032808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":684032,"data_stored":665807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":248189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":831488,"data_stored":775727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14381056,"data_stored":16070699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2879488,"data_stored":3224667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5632000,"data_stored":6161359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9269248,"data_stored":10544504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:59:43.268 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [1, 4, 6, 5] 2024-09-17T22:59:43.268 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1', '1', '4', '6', '5'] 2024-09-17T22:59:43.268 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1 1 4 6 5 2024-09-17T22:59:43.307 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:43.305+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:48548 3.c 3:30f49134:::rbd_object_map.1188a43cbaa8.000000000000001b:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 104.195488045, 2024-09-17T22:59:43.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.70308331400000001, 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 48548 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.353 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153542+0000", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153543+0000", 2024-09-17T22:59:43.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7740000000000001e-06 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153550+0000", 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.319e-06 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153564+0000", 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.449e-05 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153573+0000", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3599999999999996e-06 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153635+0000", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2339000000000001e-05 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for peered", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.153646+0000", 2024-09-17T22:59:43.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0604e-05 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854676+0000", 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.701030245 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854813+0000", 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013679799999999999 2024-09-17T22:59:43.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "waiting for subops from 6", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.854915+0000", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000102693 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "op_commit", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.855497+0000", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00058193999999999997 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856493+0000", 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00099600600000000002 2024-09-17T22:59:43.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856537+0000", 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3825999999999997e-05 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:57:59.856625+0000", 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7919999999999998e-05 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62377 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2258944~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843699646000005, 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051074646000000001, 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62377 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505330+0000", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505331+0000", 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505333+0000", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2390000000000001e-06 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505340+0000", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7839999999999998e-06 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505344+0000", 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6240000000000001e-06 2024-09-17T22:59:43.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556233+0000", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050888600999999999 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556311+0000", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8571000000000004e-05 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556405+0000", 2024-09-17T22:59:43.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3277999999999997e-05 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62378 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2271232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843678936000003, 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051277011999999997, 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.364 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62378 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505351+0000", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7e-07 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505353+0000", 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505359+0000", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1399999999999997e-06 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505364+0000", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556452+0000", 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051087403000000003 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556535+0000", 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.3248e-05 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556628+0000", 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2880000000000005e-05 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.368 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62379 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2283520~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843659141000003, 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051480593999999998, 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62379 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505371+0000", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505373+0000", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505379+0000", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9939999999999997e-06 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505383+0000", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556670+0000", 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051286381999999998 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556750+0000", 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9919000000000005e-05 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556851+0000", 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000101381 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62380 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2295808~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843639898000006, 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051674528999999997, 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62380 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.373 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505390+0000", 2024-09-17T22:59:43.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505392+0000", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0379999999999998e-06 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505398+0000", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2419999999999997e-06 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505405+0000", 2024-09-17T22:59:43.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556882+0000", 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051476800000000003 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.556959+0000", 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7280999999999994e-05 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557064+0000", 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000105516 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62381 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2308096~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843618758000005, 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05197591, 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.377 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62381 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505411+0000", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505418+0000", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.494e-06 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505425+0000", 2024-09-17T22:59:43.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0439999999999998e-06 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505430+0000", 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557097+0000", 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051667230000000001 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557170+0000", 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2613000000000004e-05 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557387+0000", 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00021725600000000001 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62382 3.2 3:47df2125:::rbd_data.1188b9dbd70.000000000000000d:23 [read 2320384~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843593669000001, 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052201869999999997, 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62382 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505436+0000", 2024-09-17T22:59:43.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505438+0000", 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.8670000000000001e-06 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505444+0000", 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.6450000000000002e-06 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505449+0000", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6160000000000004e-06 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557424+0000", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051975823999999997 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557506+0000", 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1488000000000003e-05 2024-09-17T22:59:43.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557638+0000", 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00013202999999999999 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62383 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843574325999995, 2024-09-17T22:59:43.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052255823999999999, 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62383 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505455+0000", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505456+0000", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5699999999999998e-07 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505459+0000", 2024-09-17T22:59:43.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.591e-06 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505466+0000", 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505471+0000", 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.135e-06 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557702+0000", 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052230889000000003 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557711+0000", 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2709999999999998e-06 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.390 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62384 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843552302999996, 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052265957000000002, 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62384 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505477+0000", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505478+0000", 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505480+0000", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505486+0000", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505491+0000", 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:59:43.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557737+0000", 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052245713999999999 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557743+0000", 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.584e-06 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62385 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843458162000005, 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052202941000000003, 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62385 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.395 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505572+0000", 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505575+0000", 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9960000000000002e-06 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505584+0000", 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6840000000000002e-06 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505589+0000", 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.5879999999999997e-06 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557768+0000", 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052178809999999999 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557774+0000", 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2090000000000002e-06 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62386 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843431390000006, 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052192637, 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.399 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62386 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505598+0000", 2024-09-17T22:59:43.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505599+0000", 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3600000000000004e-07 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505601+0000", 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505607+0000", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505612+0000", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7960000000000003e-06 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557787+0000", 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052175356999999999 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557791+0000", 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62387 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843410864000006, 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052186834000000001, 2024-09-17T22:59:43.403 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62387 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505619+0000", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505621+0000", 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0679999999999999e-06 2024-09-17T22:59:43.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505628+0000", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.652e-06 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505633+0000", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1889999999999998e-06 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557802+0000", 2024-09-17T22:59:43.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05216908 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557806+0000", 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.3330000000000001e-06 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62388 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843389571000003, 2024-09-17T22:59:43.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052183596999999998, 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62388 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:43.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505640+0000", 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505641+0000", 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505643+0000", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505649+0000", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6760000000000001e-06 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505655+0000", 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.293e-06 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557818+0000", 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052162899999999998 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557824+0000", 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1820000000000003e-06 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62389 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843360696000005, 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052181698999999998, 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62389 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505669+0000", 2024-09-17T22:59:43.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505672+0000", 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2960000000000001e-06 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505679+0000", 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:59:43.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505688+0000", 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5799999999999998e-06 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557844+0000", 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155524000000002 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557851+0000", 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.703e-06 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62390 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843319617000006, 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052164655999999997, 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.416 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62390 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505710+0000", 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505711+0000", 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505713+0000", 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3539999999999998e-06 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505720+0000", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1740000000000003e-06 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505726+0000", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557871+0000", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052144939000000001 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557875+0000", 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.6109999999999998e-06 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:62391 3.2 3.a484fbe2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 89.843295593999997, 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052155422, 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "reached pg", 2024-09-17T22:59:43.420 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 62391 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.505734+0000", 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507017+0000", 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012833390000000001 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507022+0000", 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0609999999999997e-06 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507031+0000", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.5459999999999997e-06 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.507037+0000", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557886+0000", 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050848649000000003 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:13.557890+0000", 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.4149999999999999e-06 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90158 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 250880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 85.958552967000003, 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051034674000000002, 2024-09-17T22:59:43.424 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90158 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390477+0000", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390479+0000", 2024-09-17T22:59:43.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390486+0000", 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390492+0000", 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4330000000000003e-06 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441288+0000", 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.050796249000000002 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441364+0000", 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6618000000000004e-05 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441511+0000", 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146933 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.428 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90159 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 85.958531339000004, 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.05126845, 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.429 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90159 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390498+0000", 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390499+0000", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390501+0000", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390508+0000", 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7340000000000002e-06 2024-09-17T22:59:43.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390513+0000", 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441543+0000", 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051029439000000003 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.432 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441620+0000", 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7477000000000004e-05 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441767+0000", 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000146701 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90160 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:43.433 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 85.958510154999999, 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051949662000000001, 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90160 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.434 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390520+0000", 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:43.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390522+0000", 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390529+0000", 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7850000000000003e-06 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.436 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390534+0000", 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.441797+0000", 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051262896000000002 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442376+0000", 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00057869900000000001 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.437 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442469+0000", 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3602000000000003e-05 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_op(client.4488.0:90161 3.2 3:46272247:::rbd_data.1188f4e2c696.0000000000000009:24 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 85.958489075000003, 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.052179706999999999, 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:43.438 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client_info": { 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client": "client.4488", 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "tid": 90161 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.439 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390541+0000", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390549+0000", 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.8860000000000004e-06 2024-09-17T22:59:43.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390556+0000", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.1080000000000004e-06 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.390561+0000", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442501+0000", 2024-09-17T22:59:43.441 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.051940312000000002 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442574+0000", 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.2854000000000004e-05 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:58:17.442720+0000", 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.000145993 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.442 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:43.443 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:43.443 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:43.444 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:43.445 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:43.442+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:43.545 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:43.543+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:43.548 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:43.548 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:43.548 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:43.548 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:43.556 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:43.646 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:43.644+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:43.701 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:43.701 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:43.702 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:43.702 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:43.702 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:43.710 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:43.751 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:43.749+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:43.847 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:43.845+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:89489 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 260096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.467310620999996, 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376972000000002, 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.856 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 89489 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385274+0000", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385275+0000", 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1209999999999999e-06 2024-09-17T22:59:43.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385278+0000", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.385281+0000", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445116+0000", 2024-09-17T22:59:43.859 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.059834819999999997 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.445193+0000", 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6996999999999995e-05 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.446651+0000", 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001458059 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91023 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.428337669000001, 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061084767999999998, 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91023 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.861 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424247+0000", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424249+0000", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.973e-06 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424256+0000", 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6529999999999997e-06 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424261+0000", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6459999999999998e-06 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485146+0000", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884757999999997 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485229+0000", 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2918999999999997e-05 2024-09-17T22:59:43.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485331+0000", 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010234300000000001 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91024 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.428316893000002, 2024-09-17T22:59:43.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061268860000000001, 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91024 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424267+0000", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424268+0000", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.65e-07 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424270+0000", 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:59:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424276+0000", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7869999999999998e-06 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424280+0000", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.814e-06 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485369+0000", 2024-09-17T22:59:43.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061088898000000003 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485420+0000", 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0976000000000001e-05 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485536+0000", 2024-09-17T22:59:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0001161 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91025 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.870 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.428298103000003, 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061440527000000002, 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91025 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.871 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424286+0000", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424287+0000", 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424288+0000", 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424294+0000", 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4110000000000004e-06 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424298+0000", 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5009999999999998e-06 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485570+0000", 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061271605999999999 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485640+0000", 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9950999999999995e-05 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485727+0000", 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6823999999999994e-05 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91026 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.428252892000003, 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061596801, 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.875 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91026 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424331+0000", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424332+0000", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7899999999999997e-07 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424337+0000", 2024-09-17T22:59:43.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3490000000000002e-06 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424349+0000", 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2913e-05 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424356+0000", 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6279999999999999e-06 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485754+0000", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397550000000002 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485814+0000", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0146000000000003e-05 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485928+0000", 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000114436 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.879 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:91027 3.1d 3:bbb5092d:::rbd_data.1188f4e2c696.0000000000000002:24 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 86.428219064000004, 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061793973000000002, 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.880 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 91027 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424365+0000", 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424366+0000", 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424368+0000", 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.882 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424373+0000", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2870000000000004e-06 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.424376+0000", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0929999999999999e-06 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.485962+0000", 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061586150999999999 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486056+0000", 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3478000000000002e-05 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:17.486159+0000", 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103411 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.884 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189924 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.723301796999998, 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061139127000000001, 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189924 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129283+0000", 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.39e-07 2024-09-17T22:59:43.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129285+0000", 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0490000000000002e-06 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129291+0000", 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2570000000000004e-06 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129296+0000", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190188+0000", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060891990999999999 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190260+0000", 2024-09-17T22:59:43.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.2033000000000006e-05 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016150800000000001 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189925 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 73728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:43.889 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.723281329000002, 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061118556999999997, 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189925 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129303+0000", 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129306+0000", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3479999999999998e-06 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129312+0000", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2609999999999999e-06 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129317+0000", 2024-09-17T22:59:43.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190201+0000", 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060884372999999999 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190265+0000", 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3751000000000007e-05 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190422+0000", 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015661399999999999 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189934 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.723259767999998, 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061233725000000003, 2024-09-17T22:59:43.894 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189934 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:43.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129325+0000", 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129327+0000", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2369999999999999e-06 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129333+0000", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.003e-06 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129338+0000", 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8019999999999999e-06 2024-09-17T22:59:43.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190452+0000", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061114333999999999 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190489+0000", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6806e-05 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190558+0000", 2024-09-17T22:59:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8979000000000005e-05 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189937 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.723239950999996, 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061397057999999997, 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.899 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189937 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129344+0000", 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129345+0000", 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129347+0000", 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0650000000000001e-06 2024-09-17T22:59:43.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129353+0000", 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6560000000000003e-06 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129358+0000", 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190592+0000", 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061234153999999999 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190659+0000", 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6381e-05 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2548999999999999e-05 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.903 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189940 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 311296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.723219962999998, 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061376722000000002, 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.904 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189940 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129364+0000", 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129365+0000", 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129367+0000", 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9240000000000001e-06 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129373+0000", 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9699999999999996e-06 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.129377+0000", 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7049999999999998e-06 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190584+0000", 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061206479000000001 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190650+0000", 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.6171000000000001e-05 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190741+0000", 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1073e-05 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189951 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722554549999998, 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061065332, 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.909 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189951 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130030+0000", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9400000000000005e-07 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130033+0000", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5459999999999998e-06 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130040+0000", 2024-09-17T22:59:43.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130045+0000", 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3580000000000002e-06 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190947+0000", 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060901046 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191011+0000", 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4197999999999998e-05 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191095+0000", 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4401999999999996e-05 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.913 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189956 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722531805000003, 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061043396, 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189956 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130053+0000", 2024-09-17T22:59:43.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.75e-07 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130055+0000", 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2290000000000002e-06 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130062+0000", 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130068+0000", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.190960+0000", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060892595000000001 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191022+0000", 2024-09-17T22:59:43.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1391999999999998e-05 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191096+0000", 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4251000000000004e-05 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189957 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:43.918 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722510671999999, 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061192518000000001, 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189957 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:43.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130074+0000", 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.77e-07 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130076+0000", 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0209999999999998e-06 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130082+0000", 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4420000000000003e-06 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130088+0000", 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2000000000000002e-06 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191120+0000", 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061032712000000003 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191184+0000", 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.3656000000000002e-05 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2109999999999998e-05 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189964 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722490555, 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061172627, 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.923 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189964 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:43.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130094+0000", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130102+0000", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3140000000000004e-06 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130109+0000", 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7769999999999997e-06 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130115+0000", 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191132+0000", 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061017476000000001 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191193+0000", 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.0538000000000002e-05 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191266+0000", 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3633000000000005e-05 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189965 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.928 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722463345000001, 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061314419000000002, 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189965 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130121+0000", 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:59:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130123+0000", 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9139999999999998e-06 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130129+0000", 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130280+0000", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015112799999999999 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191290+0000", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061009304 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191352+0000", 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2520000000000004e-05 2024-09-17T22:59:43.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191435+0000", 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3249000000000002e-05 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189973 3.1c 3:3a8f9425:::rbd_data.1188584c80a2.000000000000001c:35 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722292484, 2024-09-17T22:59:43.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061148367000000002, 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189973 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130292+0000", 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9500000000000002e-07 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130296+0000", 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1829999999999998e-06 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130304+0000", 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9889999999999992e-06 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130310+0000", 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1319999999999999e-06 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191301+0000", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.060991558000000001 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191361+0000", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9330999999999998e-05 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191440+0000", 2024-09-17T22:59:43.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9579000000000005e-05 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:189980 3.14 3:2fd0f506:::rbd_data.1188584c80a2.0000000000000022:35 [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 63.722268165999999, 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061287987000000002, 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:43.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 189980 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130316+0000", 2024-09-17T22:59:43.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130317+0000", 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130319+0000", 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130325+0000", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2890000000000003e-06 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.130330+0000", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191461+0000", 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.061130929000000001 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191523+0000", 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2281999999999998e-05 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:40.191604+0000", 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1058999999999998e-05 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:196589 3.f 3:f2e53d75:::rbd_object_map.1188400f7d88:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 58.716994163999999, 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0004686860000001, 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 196589 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135590+0000", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135592+0000", 2024-09-17T22:59:43.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.623e-06 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135600+0000", 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.028e-06 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135614+0000", 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135621+0000", 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6909999999999997e-06 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135679+0000", 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.804e-05 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:45.135689+0000", 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.133760+0000", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.99807064999999995 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134115+0000", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00035551099999999998 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.134299+0000", 2024-09-17T22:59:43.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183721 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135703+0000", 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001404244 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.135982+0000", 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027882500000000001 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136021+0000", 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9422000000000003e-05 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:46.136059+0000", 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7588e-05 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200002 3.1b 3:dea2114f:::rbd_object_map.11884120dce1:head [call lock.unlock in=10b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 27.858419718, 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0273163620000001, 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200002 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:43.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994165+0000", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994166+0000", 2024-09-17T22:59:43.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1620000000000001e-06 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994172+0000", 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2330000000000003e-06 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994181+0000", 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3190000000000001e-06 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994188+0000", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8190000000000003e-06 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994239+0000", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0865000000000002e-05 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:15.994250+0000", 2024-09-17T22:59:43.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1191e-05 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019705+0000", 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0254550520000001 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.019922+0000", 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021627500000000001 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:43.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020180+0000", 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025870299999999999 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.020983+0000", 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00080232199999999995 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021341+0000", 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000358356 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021391+0000", 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0390999999999999e-05 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.021481+0000", 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9673999999999996e-05 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:43.956 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:43.957 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:43.958 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:43.958 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:43.955+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:43.963 INFO:teuthology.orchestra.run.smithi159.stderr:set 3.1 pg_upmap_items mapping to [1->4,6->5] 2024-09-17T22:59:44.059 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:44.057+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:44.109 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.109 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:44.109 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:44.109 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.117 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:44.162 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:44.160+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:44.262 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:44.260+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:44.267 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.267 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:44.267 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:44.267 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:44.267 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.276 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:44.362 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:44.360+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:44.421 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.165876986, 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.422 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:44.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.424 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.426 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16586098100001, 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:44.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:44.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16582821099999, 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.431 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:44.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.435 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16580047299999, 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:44.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16577805399999, 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.440 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:44.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.444 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16575577399999, 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.445 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.449 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16554836899999, 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:44.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:44.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.165001754, 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.454 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.458 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.163930558, 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:44.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:44.462 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16391910300001, 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.463 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:44.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.163908126, 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:44.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:44.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:44.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16183399400001, 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:44.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:44.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16182168, 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:44.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:44.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:44.481 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16180844300001, 2024-09-17T22:59:44.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:44.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:44.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.16179761399999, 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:44.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.15983873900001, 2024-09-17T22:59:44.493 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.494 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.496 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.497 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.15981438099999, 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.498 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.499 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:44.500 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.501 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.502 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.15978516300001, 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.503 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.504 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:44.505 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:44.506 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 128.159005493, 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.507 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.508 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:44.509 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.510 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 46.540146954999997, 2024-09-17T22:59:44.511 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:44.512 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.513 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:44.514 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:44.515 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:44.516 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.517 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.518 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.520 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:44.520 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:44.517+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:44.620 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:44.618+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:44.664 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:44.665 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:44.665 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:44.721 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:44.719+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:44.775 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:44.775 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:44.775 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.783 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:44.827 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:44.825+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:44.895 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:44.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 161.03163192900001, 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.896 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:44.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.80479280200001, 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:44.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:44.905 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.80477427599999, 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.908 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:44.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:44.912 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.80475610400001, 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:44.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.80473827599999, 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.917 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:44.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:44.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 133.80442314300001, 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:44.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:44.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.926 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322598808999999, 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:44.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:44.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322589180999998, 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.931 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.935 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322579833000006, 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:44.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:44.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322569911000002, 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.940 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322556454999997, 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:44.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322546642999995, 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.949 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.953 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322516273999994, 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:44.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:44.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322503951000002, 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.958 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.960 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:44.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.962 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322494444, 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.963 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:44.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322484403000004, 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.968 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:44.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.971 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322475119000003, 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:44.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:44.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322465625000007, 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:44.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:44.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:44.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322454356999998, 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.982 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:44.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.984 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.986 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 65.322444951999998, 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:44.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:44.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:44.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:44.991 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:44.991 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:59:44.992 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:44.990+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:44.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:45.093 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:45.091+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:45.137 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:45.138 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:45.138 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:45.146 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:45.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:59:45.231 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:45.229+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:45.291 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:45.291 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:45.291 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:45.294 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:45.292+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:45.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:45.308 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:45.308 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:45.324 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:45.325 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:45.340 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:45.340 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:45.356 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:45.356 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:45.372 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:45.372 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:45.388 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:45.388 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:45.395 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:45.393+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:45.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:45.404 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:45.420 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:45.420 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:45.445 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:45.446 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:45.454 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:45.496 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:45.494+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:45.574 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:45.574 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:45.583 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:45.597 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:45.595+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:45.685 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:45.685 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:45.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:45.737 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:45.735+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:45.797 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:45.797 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:45.799 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:45.797+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:45.805 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:59:45.900 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:45.897+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:45.949 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:45.949 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:45.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:59:45.969 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is set 2024-09-17T22:59:46.001 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:45.999+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:46.086 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.086 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.095 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:59:46.138 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-09-17T22:59:46.136+0000 7f481eec7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33554) UID: 0 2024-09-17T22:59:46.199 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.199 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.202 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:46.200+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:46.207 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:59:46.303 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:46.301+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:46.351 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.351 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.359 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:59:46.404 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:46.402+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:46.503 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.504 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.505 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:46.503+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:46.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:59:46.606 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-09-17T22:59:46.603+0000 7fc0857f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33568) UID: 0 2024-09-17T22:59:46.616 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.616 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.625 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:59:46.708 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:46.706+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:46.767 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.768 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.776 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:59:46.817 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:46.815+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:46.909 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-09-17T22:59:46.907+0000 7f0ad92ac640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 55294) UID: 0 2024-09-17T22:59:46.920 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:46.920 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:46.928 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:59:47.010 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:47.008+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:47.084 INFO:teuthology.orchestra.run.smithi159.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:47.084 INFO:teuthology.orchestra.run.smithi159.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:47.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:59:47.138 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-17T22:59:47.136+0000 7f839751b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33184) UID: 0 2024-09-17T22:59:47.200 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:47.200 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:47.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:59:47.212 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:47.210+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:47.313 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:47.311+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:47.352 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:47.353 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:47.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:59:47.413 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:47.412+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:47.432 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:59:47.505 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:47.505 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:47.514 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:59:47.515 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-09-17T22:59:47.512+0000 7f483a821640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33574) UID: 0 2024-09-17T22:59:47.552 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0: {} 2024-09-17T22:59:47.552 INFO:teuthology.orchestra.run.smithi159.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:59:47.568 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1: {} 2024-09-17T22:59:47.569 INFO:teuthology.orchestra.run.smithi159.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:59:47.584 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2: {} 2024-09-17T22:59:47.584 INFO:teuthology.orchestra.run.smithi159.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:59:47.600 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3: {} 2024-09-17T22:59:47.600 INFO:teuthology.orchestra.run.smithi159.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:59:47.615 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:47.613+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:47.616 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4: {} 2024-09-17T22:59:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:59:47.620 INFO:teuthology.orchestra.run.smithi171.stdout:ERROR: (22) Invalid argument 2024-09-17T22:59:47.620 INFO:teuthology.orchestra.run.smithi171.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:59:47.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:59:47.632 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5: {} 2024-09-17T22:59:47.632 INFO:teuthology.orchestra.run.smithi159.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:59:47.648 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6: {} 2024-09-17T22:59:47.649 INFO:teuthology.orchestra.run.smithi159.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:59:47.665 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7: {} 2024-09-17T22:59:47.665 INFO:teuthology.orchestra.run.smithi159.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:59:47.716 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-17T22:59:47.714+0000 7fbef292d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44289) UID: 0 2024-09-17T22:59:47.773 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:47.773 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:47.773 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:47.773 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:47.773 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:47.781 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:59:47.822 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:47.820+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:47.917 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-17T22:59:47.915+0000 7fbcc9c60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33188) UID: 0 2024-09-17T22:59:47.926 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:47.926 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:47.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:47.926 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:47.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16054 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.67045848800001, 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050537054999999997, 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16054 2024-09-17T22:59:47.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252790+0000", 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252792+0000", 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8560000000000001e-06 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252798+0000", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252801+0000", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1410000000000001e-06 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.302973+0000", 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050171976 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303052+0000", 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9259999999999997e-05 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303327+0000", 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027483199999999997 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16055 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.67044248299999, 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050796527000000001, 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16055 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.932 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252806+0000", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252819+0000", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3206e-05 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252826+0000", 2024-09-17T22:59:47.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000002e-06 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252831+0000", 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9590000000000003e-06 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303356+0000", 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050524697 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303400+0000", 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3676e-05 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303602+0000", 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020241700000000001 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.936 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16056 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.670409713, 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050966735999999999, 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16056 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.937 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252838+0000", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252839+0000", 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.51e-07 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.938 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252841+0000", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.951e-06 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252848+0000", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7299999999999999e-06 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252858+0000", 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0029e-05 2024-09-17T22:59:47.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303632+0000", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050774183000000001 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303716+0000", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4628000000000004e-05 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303805+0000", 2024-09-17T22:59:47.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8764000000000006e-05 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16057 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.670381975, 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051213868000000003, 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.941 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16057 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252866+0000", 2024-09-17T22:59:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252867+0000", 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252869+0000", 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252877+0000", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9659999999999996e-06 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252882+0000", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0379999999999999e-06 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303834+0000", 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050952099000000001 2024-09-17T22:59:47.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.303882+0000", 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8408999999999997e-05 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304080+0000", 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00019780700000000001 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.945 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16058 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.67035955599999, 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051578323000000002, 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16058 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252889+0000", 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252891+0000", 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252899+0000", 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2269999999999993e-06 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252904+0000", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304112+0000", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051207376999999998 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304190+0000", 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8422000000000002e-05 2024-09-17T22:59:47.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304467+0000", 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027690400000000002 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16059 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.670337276, 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051836649999999998, 2024-09-17T22:59:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16059 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.951 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.252911+0000", 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253087+0000", 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000175737 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.952 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253089+0000", 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253103+0000", 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.382e-05 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253110+0000", 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.207e-06 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.953 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304484+0000", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051374325999999998 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304528+0000", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4261000000000002e-05 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304748+0000", 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000219428 2024-09-17T22:59:47.954 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16060 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.670129871, 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051830971000000003, 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.955 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16060 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.956 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253118+0000", 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253635+0000", 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00051661800000000005 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253638+0000", 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7930000000000002e-06 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.957 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253646+0000", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5029999999999997e-06 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253657+0000", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.084e-05 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304778+0000", 2024-09-17T22:59:47.958 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051120956000000002 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304860+0000", 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1773000000000005e-05 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304949+0000", 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.9487999999999994e-05 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.959 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16061 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66958325600001, 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051541363, 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.960 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16061 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.253665+0000", 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.961 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254713+0000", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001048457 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254716+0000", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7130000000000001e-06 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254724+0000", 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.371e-06 2024-09-17T22:59:47.962 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254729+0000", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.357e-06 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.304977+0000", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050247977999999999 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305029+0000", 2024-09-17T22:59:47.963 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2500000000000002e-05 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305206+0000", 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000176987 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16062 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.964 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66851206000001, 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050780087000000002, 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16062 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.965 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254736+0000", 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.96e-07 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.966 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254738+0000", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.296e-06 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254741+0000", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254744+0000", 2024-09-17T22:59:47.967 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.74e-06 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305236+0000", 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050492039000000002 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305316+0000", 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9705999999999999e-05 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.968 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305516+0000", 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00020029600000000001 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16063 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66850060499999, 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050992040000000002, 2024-09-17T22:59:47.969 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16063 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.970 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.971 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.971 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:47.971 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254748+0000", 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254749+0000", 2024-09-17T22:59:47.972 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254752+0000", 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.298e-06 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254755+0000", 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.973 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305531+0000", 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050776226000000001 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305575+0000", 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4107999999999998e-05 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305740+0000", 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016451000000000001 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.974 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16064 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.668489628, 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051362805999999997, 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16064 2024-09-17T22:59:47.975 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.254759+0000", 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.976 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256711+0000", 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001952098 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256714+0000", 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256721+0000", 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5830000000000001e-06 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.977 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256725+0000", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2010000000000001e-06 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305770+0000", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049044507000000001 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.305850+0000", 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:59:47.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306121+0000", 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000271697 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16068 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66641549600001, 2024-09-17T22:59:47.979 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050522782000000002, 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16068 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256833+0000", 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.981 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256834+0000", 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3629999999999999e-06 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256838+0000", 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6279999999999998e-06 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256842+0000", 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7519999999999998e-06 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.982 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.306981+0000", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050138978000000001 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307063+0000", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.2486999999999998e-05 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307355+0000", 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00029222699999999999 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.983 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16069 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66640318200001, 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050879132000000001, 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.984 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16069 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.985 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256845+0000", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4499999999999998e-07 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256846+0000", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0249999999999999e-06 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256849+0000", 2024-09-17T22:59:47.986 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.247e-06 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256853+0000", 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.63e-06 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307402+0000", 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050549152999999999 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.987 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307479+0000", 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.6412e-05 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307724+0000", 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00024541999999999999 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16070 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.988 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66638994499999, 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051130621000000001, 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16070 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.989 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256858+0000", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.72e-07 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.990 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256860+0000", 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.195e-06 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256863+0000", 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256866+0000", 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.711e-06 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307761+0000", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050895146000000002 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307815+0000", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3517000000000003e-05 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.307989+0000", 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017424899999999999 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16071 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.666379116, 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.051510275000000001, 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.993 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16071 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.256869+0000", 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.994 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258711+0000", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018424660000000001 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258716+0000", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0810000000000004e-06 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258726+0000", 2024-09-17T22:59:47.995 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0023e-05 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258730+0000", 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308025+0000", 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049294655 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.996 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308103+0000", 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7985999999999995e-05 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.308379+0000", 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00027624799999999999 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.997 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16139 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66442024099999, 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050614780999999998, 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16139 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.998 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258828+0000", 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4700000000000001e-07 2024-09-17T22:59:47.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258831+0000", 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258838+0000", 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.542e-06 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.000 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258843+0000", 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309275+0000", 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050431932999999998 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309355+0000", 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9730999999999999e-05 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.001 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309443+0000", 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7551000000000003e-05 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16140 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1253376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.664395883, 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050723546000000001, 2024-09-17T22:59:48.002 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16140 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258852+0000", 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258853+0000", 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3200000000000005e-07 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.004 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258861+0000", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9559999999999997e-06 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258868+0000", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7389999999999999e-06 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258874+0000", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.005 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309471+0000", 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050596869000000003 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309516+0000", 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5266000000000003e-05 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309576+0000", 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9753000000000002e-05 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.006 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16141 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.66436666499999, 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050897385000000003, 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16141 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.007 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.258881+0000", 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259585+0000", 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00070303100000000001 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259589+0000", 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4290000000000003e-06 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.009 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259600+0000", 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1367e-05 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259617+0000", 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6215e-05 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309607+0000", 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049990160999999998 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.010 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309685+0000", 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8769000000000004e-05 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.309779+0000", 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3412999999999995e-05 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.011 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:16144 3.10 3:0a4c5631:::rbd_data.1188b0ea165c.0000000000000001:a [read 1449984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 131.663586995, 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.050577487999999997, 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 16144 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.012 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.259661+0000", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260712+0000", 2024-09-17T22:59:48.013 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010504729999999999 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260716+0000", 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9269999999999998e-06 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260726+0000", 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0926e-05 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.014 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.260735+0000", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5439999999999999e-06 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310134+0000", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049398689000000003 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310183+0000", 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.969e-05 2024-09-17T22:59:48.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:36.310239+0000", 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.5238999999999997e-05 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197922 3.10 3:0cb9034a:::rbd_object_map.1188fd9d2506:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:48.016 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.044728456999998, 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0183586249999999, 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197922 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:48.017 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878520+0000", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878521+0000", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3120000000000001e-06 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878527+0000", 2024-09-17T22:59:48.018 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7880000000000003e-06 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878538+0000", 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1141999999999999e-05 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878544+0000", 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878602+0000", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8375999999999998e-05 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.878614+0000", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1387e-05 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879251+0000", 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0006368649999999 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.020 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:58.879264+0000", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2869e-05 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895058+0000", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0157942520000001 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895247+0000", 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00018958000000000001 2024-09-17T22:59:48.021 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 5", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.895465+0000", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00021720299999999999 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896374+0000", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00090950099999999995 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896737+0000", 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00036255400000000001 2024-09-17T22:59:48.022 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896782+0000", 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5345e-05 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:59.896878+0000", 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6288999999999999e-05 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.023 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.024 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.024 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:59:48.026 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:59:48.026 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-17T22:59:48.023+0000 7f57be99a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33193) UID: 0 2024-09-17T22:59:48.069 INFO:teuthology.orchestra.run.smithi159.stdout:All operations completed A-OK! 2024-09-17T22:59:48.084 DEBUG:teuthology.parallel:result is None 2024-09-17T22:59:48.085 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T22:59:48.094 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T22:59:48.133 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T22:59:48.170 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.171 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:48.171 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:48.171 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:59:48.284 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:48.284 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:48.284 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.293 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:59:48.398 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.398 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:48.398 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:48.398 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 164.53468580000001, 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31500035799999998, 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4213", 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2236308530", 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 1 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.399 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860601+0000", 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.400 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860603+0000", 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5510000000000001e-06 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860611+0000", 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.4090000000000003e-06 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860622+0000", 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1054e-05 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.401 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860631+0000", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.5120000000000008e-06 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860699+0000", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8112000000000001e-05 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:03.860719+0000", 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9528999999999999e-05 2024-09-17T22:59:48.402 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174109+0000", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.31339028400000002 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174273+0000", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016416400000000001 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 0", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.174439+0000", 2024-09-17T22:59:48.403 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000166186 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175128+0000", 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00068826800000000002 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175530+0000", 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00040286 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.404 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175570+0000", 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0055999999999997e-05 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:04.175602+0000", 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.1372999999999999e-05 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.405 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7066 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 4980736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.307846673, 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048119201, 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7066 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.406 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087441+0000", 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:59:48.407 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087443+0000", 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.934e-06 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087449+0000", 2024-09-17T22:59:48.408 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7289999999999997e-06 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087454+0000", 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135420+0000", 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047966234000000003 2024-09-17T22:59:48.409 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135477+0000", 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6971000000000001e-05 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135560+0000", 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3077000000000004e-05 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7067 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5177344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.30782814700001, 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0483168, 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7067 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.411 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087459+0000", 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8299999999999998e-07 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.412 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087461+0000", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087467+0000", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.75e-06 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087472+0000", 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7190000000000001e-06 2024-09-17T22:59:48.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135597+0000", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048124901999999997 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135651+0000", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4394000000000001e-05 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135776+0000", 2024-09-17T22:59:48.414 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012469099999999999 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7068 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.307809975, 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048425731, 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.415 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7068 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087477+0000", 2024-09-17T22:59:48.416 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087478+0000", 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.58e-07 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087480+0000", 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.417 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087485+0000", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087489+0000", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135805+0000", 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048315782000000002 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.418 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135850+0000", 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4966000000000003e-05 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135903+0000", 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3233999999999999e-05 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.419 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7069 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.30779214699999, 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048530957999999999, 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7069 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.420 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087495+0000", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087776+0000", 2024-09-17T22:59:48.421 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000281165 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087780+0000", 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6890000000000002e-06 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087792+0000", 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.181e-05 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.422 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087798+0000", 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135930+0000", 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048131759000000003 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.135974+0000", 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3999999999999999e-05 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.423 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136026+0000", 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1826000000000002e-05 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:7070 3.d 3:b6990f07:::rbd_data.1188b3deeabb.0000000000000000:8 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 137.307477014, 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048328031, 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.424 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 7070 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.425 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.087810+0000", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088239+0000", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00042893400000000001 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088244+0000", 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.673e-06 2024-09-17T22:59:48.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088513+0000", 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00026935199999999999 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.088521+0000", 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.5569999999999998e-06 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.427 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136048+0000", 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047527393000000001 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136090+0000", 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1462999999999999e-05 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:57:31.136138+0000", 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8659000000000003e-05 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.428 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183078 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11010048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825652680000005, 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048001307, 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.429 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183078 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569635+0000", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569636+0000", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.184e-06 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569639+0000", 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:59:48.431 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569642+0000", 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617544+0000", 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.047901965999999997 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.432 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617586+0000", 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2719999999999998e-05 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617636+0000", 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9591e-05 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183080 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11046912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.433 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825643052000004, 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048127649000000002, 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183080 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.434 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569644+0000", 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.435 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569646+0000", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1370000000000001e-06 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569648+0000", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.92e-06 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569651+0000", 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:59:48.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617663+0000", 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048012020000000002 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617713+0000", 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0244000000000002e-05 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.437 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617772+0000", 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8733000000000002e-05 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183082 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11083776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825633703999998, 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048242158, 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.438 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183082 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.439 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569654+0000", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.05e-07 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569655+0000", 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.494e-06 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.440 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569658+0000", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569661+0000", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.5830000000000001e-06 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617802+0000", 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048140897000000002 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.441 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617844+0000", 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.2412e-05 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617896+0000", 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1656999999999997e-05 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.442 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183084 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11120640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825623781999994, 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048362043, 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183084 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.443 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569663+0000", 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569664+0000", 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.444 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569667+0000", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3050000000000001e-06 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569671+0000", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9829999999999998e-06 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569674+0000", 2024-09-17T22:59:48.445 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.021e-06 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617920+0000", 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048246353999999998 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.617966+0000", 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5364999999999997e-05 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.446 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618026+0000", 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9812999999999998e-05 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183086 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11141120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825610326000003, 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048464436999999999, 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.447 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183086 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.448 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569677+0000", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.17e-07 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569679+0000", 2024-09-17T22:59:48.449 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5030000000000001e-06 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569682+0000", 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569684+0000", 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3489999999999999e-06 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618051+0000", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048366601000000002 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618091+0000", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0877999999999997e-05 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618141+0000", 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9965e-05 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.451 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183088 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11177984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825600514000001, 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048681505999999999, 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.452 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183088 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.453 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569687+0000", 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569688+0000", 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1230000000000001e-06 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.454 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569691+0000", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9210000000000001e-06 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569694+0000", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618184+0000", 2024-09-17T22:59:48.455 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048490048000000001 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618270+0000", 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.6550000000000003e-05 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618368+0000", 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8163000000000002e-05 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.456 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183090 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11214848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825570145, 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048753959999999999, 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.457 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183090 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.458 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569717+0000", 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569719+0000", 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.84e-06 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569723+0000", 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9940000000000002e-06 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.459 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569726+0000", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6649999999999999e-06 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618382+0000", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048655756000000001 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618422+0000", 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9755999999999997e-05 2024-09-17T22:59:48.460 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618471+0000", 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9573999999999997e-05 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183092 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11251712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825557821999993, 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048943859999999999, 2024-09-17T22:59:48.461 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183092 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.462 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569729+0000", 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569730+0000", 2024-09-17T22:59:48.463 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569731+0000", 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1429999999999999e-06 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569734+0000", 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.898e-06 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.464 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569736+0000", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.4679999999999999e-06 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618502+0000", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048765214000000001 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618583+0000", 2024-09-17T22:59:48.465 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.1335000000000006e-05 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618673+0000", 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0464999999999995e-05 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183094 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11272192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:48.466 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825548315000006, 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049056093000000002, 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183094 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:48.467 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569739+0000", 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.468 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569740+0000", 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.099e-06 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569743+0000", 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8420000000000001e-06 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569746+0000", 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9160000000000001e-06 2024-09-17T22:59:48.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618700+0000", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.048953982 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618743+0000", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3430999999999998e-05 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618795+0000", 2024-09-17T22:59:48.470 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.1610999999999998e-05 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183096 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11309056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825538273999996, 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049250243999999999, 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.471 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183096 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.472 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569749+0000", 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.04e-07 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569750+0000", 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1340000000000001e-06 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.473 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569753+0000", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8320000000000002e-06 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569756+0000", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.491e-06 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618827+0000", 2024-09-17T22:59:48.474 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049071458999999998 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618908+0000", 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0451999999999995e-05 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.618999+0000", 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1671999999999999e-05 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.475 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183097 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11345920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825528989999995, 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049360599999999998, 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183097 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.476 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.477 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569758+0000", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569760+0000", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569763+0000", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9119999999999998e-06 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.478 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569765+0000", 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.542e-06 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619027+0000", 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049261584999999997 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619069+0000", 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1999000000000002e-05 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.479 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619119+0000", 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0148999999999997e-05 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183098 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11382784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825519495999998, 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.04955217, 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.480 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183098 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.481 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569768+0000", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569769+0000", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569773+0000", 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7450000000000001e-06 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.482 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569776+0000", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.41e-06 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619150+0000", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049374052000000002 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619231+0000", 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0786999999999997e-05 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.483 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619320+0000", 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8817e-05 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183099 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11403264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825508228000004, 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049644656000000002, 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.484 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183099 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:48.485 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569779+0000", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.36e-07 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569781+0000", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2810000000000001e-06 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569784+0000", 2024-09-17T22:59:48.486 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.943e-06 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569786+0000", 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3750000000000001e-06 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619334+0000", 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049548091000000002 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.487 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619374+0000", 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0048e-05 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619424+0000", 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9682000000000003e-05 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:183100 3.16 3:691c5b7d:::rbd_data.1188b7e16ab4.0000000000000007:34 [read 11440128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 68.825498823000004, 2024-09-17T22:59:48.488 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.049836403000000001, 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 183100 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.489 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569788+0000", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569789+0000", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1e-07 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569790+0000", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1090000000000001e-06 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.490 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.569793+0000", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.604984+0000", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.035191262000000001 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619455+0000", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.014470779 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619534+0000", 2024-09-17T22:59:48.491 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9515e-05 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:39.619625+0000", 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0628e-05 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.492 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.493 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:59:48.597 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.597 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:48.597 INFO:teuthology.orchestra.run.smithi171.stdout: "num_ops": 0 2024-09-17T22:59:48.597 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.606 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:59:48.708 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.708 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [], 2024-09-17T22:59:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "complaint_time": 30, 2024-09-17T22:59:48.709 INFO:teuthology.orchestra.run.smithi171.stdout: "num_blocked_ops": 0 2024-09-17T22:59:48.709 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.717 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:59:48.822 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "size": 20, 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 600, 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "ops": [ 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197609 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.607708762999998, 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65382346000000002, 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.823 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197609 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211293+0000", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211294+0000", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6360000000000001e-06 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211300+0000", 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6380000000000001e-06 2024-09-17T22:59:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211311+0000", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1643e-05 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211317+0000", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3199999999999999e-06 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211366+0000", 2024-09-17T22:59:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9391999999999997e-05 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.211373+0000", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864794+0000", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.65342135000000001 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864918+0000", 2024-09-17T22:59:48.827 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012307900000000001 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.865116+0000", 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000198486 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4225.0:8 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.602599679000001, 2024-09-17T22:59:48.828 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64811945599999998, 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "reached pg", 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4225", 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.171:0/2094802130", 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 8 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.829 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216402+0000", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1500000000000004e-07 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216407+0000", 2024-09-17T22:59:48.830 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.34e-06 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216416+0000", 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1020000000000003e-06 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216420+0000", 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6070000000000001e-06 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.831 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216467+0000", 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6322999999999997e-05 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for peered", 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.216474+0000", 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.832 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864427+0000", 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.64795386300000002 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.864521+0000", 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.3636999999999998e-05 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "MOSDPGPush(3.2 107/105 [PushOp(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d, version: 66'593, data_included: [0~65536,98304~114688,245760~114688,393216~114688,540672~114688,688128~114688,835584~114688,983040~114688,1130496~114688,1277952~65536], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:4488a947:::rbd_data.1188b5c0f048.0000000000000000:1d@66'593, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1343488, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:48.833 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 54.335962420000001, 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0025046140000000001, 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.834 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.483039+0000", 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484021+0000", 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098191899999999993 2024-09-17T22:59:48.835 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484036+0000", 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5254e-05 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484049+0000", 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2877000000000001e-05 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.836 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484154+0000", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000104596 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.484168+0000", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.3983e-05 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:54.485544+0000", 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001375614 2024-09-17T22:59:48.837 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:197919 3.c 3:35776921:::rbd_header.1188fd9d2506:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 50.946121411, 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0043199579999999996, 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.838 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 197919 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872880+0000", 2024-09-17T22:59:48.839 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872882+0000", 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7940000000000001e-06 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872889+0000", 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.840 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872904+0000", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4833e-05 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.872916+0000", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1612e-05 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873068+0000", 2024-09-17T22:59:48.841 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015288000000000001 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.873076+0000", 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4250000000000001e-06 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874010+0000", 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093364399999999997 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.842 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874109+0000", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9436000000000001e-05 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.874550+0000", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00044139 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.876088+0000", 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001537196 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.843 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877134+0000", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010468249999999999 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877169+0000", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.5080000000000003e-05 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:58:57.877200+0000", 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0431999999999999e-05 2024-09-17T22:59:48.844 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198354 3.1c 3:3b499c47:::rbd_object_map.118865494b0c:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 45.907508397999997, 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.003274711, 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.845 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198354 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911493+0000", 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.846 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911495+0000", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911500+0000", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2780000000000001e-06 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911511+0000", 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0825000000000001e-05 2024-09-17T22:59:48.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911519+0000", 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.9060000000000002e-06 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911564+0000", 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:48.848 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.911569+0000", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912558+0000", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00098939399999999999 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912657+0000", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8766999999999998e-05 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:48.849 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.912848+0000", 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000191817 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914633+0000", 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0017840530000000001 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914669+0000", 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.850 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914724+0000", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4706000000000001e-05 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:02.914768+0000", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.354e-05 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198524 3.1d 3:ba9a9082:::rbd_header.1188bf84e0ea:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:48.851 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.90574247, 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0045200750000000001, 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198524 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.852 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913259+0000", 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913261+0000", 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2259999999999999e-06 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.853 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913271+0000", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.8840000000000006e-06 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913292+0000", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.0696e-05 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913303+0000", 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.1048e-05 2024-09-17T22:59:48.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913376+0000", 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3312999999999994e-05 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.913383+0000", 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1879999999999996e-06 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.855 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916195+0000", 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028117950000000002 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916296+0000", 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010144100000000001 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.916608+0000", 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000311236 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.856 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917351+0000", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00074336499999999995 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917686+0000", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000334741 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917751+0000", 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.4789000000000006e-05 2024-09-17T22:59:48.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.917779+0000", 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8353000000000001e-05 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198527 3.16 3:6bc78120:::rbd_object_map.1188bf84e0ea:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 44.900306172000001, 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022866509999999998, 2024-09-17T22:59:48.858 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198527 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.859 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:48.860 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918695+0000", 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.861 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918696+0000", 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.001e-06 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918702+0000", 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7409999999999998e-06 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918721+0000", 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.9065000000000001e-05 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.862 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918728+0000", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.182e-06 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918799+0000", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0729000000000007e-05 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.918854+0000", 2024-09-17T22:59:48.863 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4861999999999998e-05 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.919008+0000", 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015435500000000001 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920892+0000", 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0018838329999999999 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.864 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920920+0000", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.834e-05 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920954+0000", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:03.920982+0000", 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8113999999999999e-05 2024-09-17T22:59:48.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.6.0:1512 3.10 e120/114 3:0d34ddd0:::rbd_data.1188f4e2c696.0000000000000008:24 v 120'640, mlcod=120'636)", 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.875605686999997, 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002306256, 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.866 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943396+0000", 2024-09-17T22:59:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3e-07 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943402+0000", 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1e-06 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943413+0000", 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0156999999999999e-05 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.868 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943419+0000", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943481+0000", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.1999000000000001e-05 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.943508+0000", 2024-09-17T22:59:48.869 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.6985999999999999e-05 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945663+0000", 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0021548990000000001 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.945702+0000", 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8847999999999998e-05 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.870 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:198974 3.16 3:6ed119e5:::rbd_object_map.1188f4e2c696:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 41.871406628999999, 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002422291, 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 198974 2024-09-17T22:59:48.871 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947595+0000", 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.872 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947596+0000", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.15e-06 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947601+0000", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4569999999999996e-06 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947614+0000", 2024-09-17T22:59:48.873 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2354999999999999e-05 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947621+0000", 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947691+0000", 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.9972e-05 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947797+0000", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106174 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.947980+0000", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000183239 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949061+0000", 2024-09-17T22:59:48.875 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010806329999999999 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949937+0000", 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00087634900000000001 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.949978+0000", 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1044000000000002e-05 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.876 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:06.950017+0000", 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.8587000000000002e-05 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_repop(osd.0.0:4100 3.1 e121/114 3:8761ec16:::rbd_data.1188b9dbd70.000000000000000c:23 v 121'1214, mlcod=120'1210)", 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.875667593000003, 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0026870119999999999, 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.877 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.878 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943334+0000", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1099999999999995e-07 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943342+0000", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.7170000000000008e-06 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943351+0000", 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2990000000000002e-06 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.879 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943361+0000", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.6779999999999995e-06 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943436+0000", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4790999999999996e-05 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.943465+0000", 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.9144000000000001e-05 2024-09-17T22:59:48.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.945966+0000", 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002500848 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:07.946021+0000", 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4824000000000002e-05 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.881 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:199302 3.11 3:8a3bf54d:::rbd_object_map.11883e449d95:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 40.752304418999998, 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0024791729999999999, 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "started", 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 199302 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.882 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066697+0000", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066698+0000", 2024-09-17T22:59:48.883 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.1900000000000001e-07 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066702+0000", 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.4889999999999997e-06 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066720+0000", 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7640999999999999e-05 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.884 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066725+0000", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066766+0000", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1007000000000002e-05 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.066814+0000", 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7392999999999998e-05 2024-09-17T22:59:48.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:08.069176+0000", 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0023624449999999999 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200096 3.11 3:8e35b8be:::rbd_data.11884120dce1.000000000000000f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.733590266, 2024-09-17T22:59:48.886 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022879160000000001, 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200096 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:48.887 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085411+0000", 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085412+0000", 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0559999999999999e-06 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.888 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085435+0000", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.3000999999999999e-05 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085451+0000", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.5899000000000001e-05 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085459+0000", 2024-09-17T22:59:48.889 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.3990000000000004e-06 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085566+0000", 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000106505 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085717+0000", 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00015122599999999999 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.890 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.085826+0000", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010882699999999999 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.086549+0000", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00072324900000000003 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087552+0000", 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0010025909999999999 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.891 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087652+0000", 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010069300000000001 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.087699+0000", 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.6470000000000001e-05 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200163 3.11 3:8ca7bd04:::rbd_header.1188b5c0f048:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 31.686354807000001, 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022726589999999998, 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200163 2024-09-17T22:59:48.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.894 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132647+0000", 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132649+0000", 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:59:48.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132654+0000", 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.897 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132657+0000", 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.106e-06 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132891+0000", 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000234212 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.132953+0000", 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.2217000000000004e-05 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.898 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133128+0000", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00017510200000000001 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.133539+0000", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00041042999999999999 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134734+0000", 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0011958190000000001 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.899 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134863+0000", 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000128555 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:17.134919+0000", 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6168999999999997e-05 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200744 3.11 3:8e4115fb:::rbd_data.1188d23b4ac7.000000000000000d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.875296731999999, 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002277519, 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200744 2024-09-17T22:59:48.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.902 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943705+0000", 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.0999999999999997e-07 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943716+0000", 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.0552e-05 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943729+0000", 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.2553e-05 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.903 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943736+0000", 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.3350000000000001e-06 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.904 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943784+0000", 2024-09-17T22:59:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7678e-05 2024-09-17T22:59:48.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943862+0000", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.8288999999999995e-05 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943933+0000", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0761999999999998e-05 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944996+0000", 2024-09-17T22:59:48.907 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001062849 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945883+0000", 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00088707900000000004 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945939+0000", 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6242999999999997e-05 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.908 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945982+0000", 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3269000000000002e-05 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:200745 3.11 3:8e06f6ab:::rbd_data.1188d23b4ac7.0000000000000013:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 23.875247181999999, 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002340698, 2024-09-17T22:59:48.909 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 200745 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.910 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943754+0000", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943755+0000", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7000000000000005e-07 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943758+0000", 2024-09-17T22:59:48.911 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0280000000000001e-06 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943765+0000", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.0929999999999997e-06 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.943771+0000", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9179999999999999e-06 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944323+0000", 2024-09-17T22:59:48.912 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00055252999999999997 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944465+0000", 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00014123799999999999 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 2", 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.944593+0000", 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00012883099999999999 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.913 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.945529+0000", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093524199999999997 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946020+0000", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00049166900000000003 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946051+0000", 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.0681e-05 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.914 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:24.946095+0000", 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.3897999999999999e-05 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:201739 3.d 3:b1d2de87:::rbd_data.11883d082c32.0000000000000007:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 17.860627178000001, 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002892113, 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.915 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 201739 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.916 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958374+0000", 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958375+0000", 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4300000000000003e-07 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958377+0000", 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1110000000000002e-06 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958383+0000", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8660000000000002e-06 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958388+0000", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3550000000000004e-06 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958438+0000", 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.9636999999999999e-05 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.918 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958509+0000", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.1336999999999998e-05 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 7", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.958565+0000", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6437000000000002e-05 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.959484+0000", 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00091845700000000002 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.919 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.960546+0000", 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00106214 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.961218+0000", 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00067234900000000004 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.920 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:30.961266+0000", 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7880999999999998e-05 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:202360 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:34.917551+0000", 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 13.901450423, 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0028325609999999999, 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.921 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 202360 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917551+0000", 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.922 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917551+0000", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917553+0000", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.7889999999999999e-06 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917560+0000", 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 7.4089999999999997e-06 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.923 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917575+0000", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.4759999999999999e-05 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917584+0000", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.8480000000000007e-06 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917641+0000", 2024-09-17T22:59:48.924 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.6956999999999997e-05 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917698+0000", 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.7188000000000002e-05 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.925 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.917858+0000", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00016027200000000001 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.920032+0000", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002173445 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.920289+0000", 2024-09-17T22:59:48.926 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00025725900000000002 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.920343+0000", 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.3939e-05 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:34.920383+0000", 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.0695000000000003e-05 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:203453 3.5 3:a7aa7c8d:::rbd_object_map.118834c4f370:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:39.873127+0000", 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.9458747029999994, 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0089345999999999991, 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.928 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 203453 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873127+0000", 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.929 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873127+0000", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873129+0000", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.1519999999999999e-06 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873139+0000", 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.9850000000000001e-06 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.930 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873155+0000", 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 1.6543999999999999e-05 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873163+0000", 2024-09-17T22:59:48.931 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.0320000000000003e-06 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873231+0000", 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.7764999999999999e-05 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.932 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op must wait for map", 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.873235+0000", 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.318e-06 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.875465+0000", 2024-09-17T22:59:48.933 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0022294110000000002 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.875569+0000", 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000103825 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 3", 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.875756+0000", 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000187011 2024-09-17T22:59:48.934 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.876574+0000", 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00081800199999999997 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.881923+0000", 2024-09-17T22:59:48.935 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0053497029999999999 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.882011+0000", 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 8.7664000000000006e-05 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.936 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.882061+0000", 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.0188000000000001e-05 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:203544 3.1d 3:bc896702:::rbd_data.118834c4f370.0000000000000011:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:39.941732+0000", 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8772689170000003, 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.002860296, 2024-09-17T22:59:48.937 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 203544 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.938 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941732+0000", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941732+0000", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941733+0000", 2024-09-17T22:59:48.939 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941736+0000", 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.8930000000000001e-06 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941743+0000", 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 6.8340000000000001e-06 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.940 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941748+0000", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.4249999999999997e-06 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.942481+0000", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00073308399999999995 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.942684+0000", 2024-09-17T22:59:48.941 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000203079 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.942817+0000", 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00013226399999999999 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.944283+0000", 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.001466347 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.942 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.944504+0000", 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000221144 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.944553+0000", 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.8909000000000002e-05 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.944593+0000", 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9730000000000001e-05 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.943 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "description": "osd_op(client.4488.0:203547 3.1d 3:bd6191a1:::rbd_data.118834c4f370.0000000000000032:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "initiated_at": "2024-09-17T22:59:39.941757+0000", 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "age": 8.8772442819999995, 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0035722050000000002, 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "continuous": false, 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "type_data": { 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_info": { 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client": "client.4488", 2024-09-17T22:59:48.944 INFO:teuthology.orchestra.run.smithi171.stdout: "client_addr": "172.21.15.159:0/2812219962", 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "tid": 203547 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "events": [ 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "initiated", 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941757+0000", 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "header_read", 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941757+0000", 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0 2024-09-17T22:59:48.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "throttled", 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941757+0000", 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "all_read", 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941760+0000", 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 2.2060000000000001e-06 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "dispatched", 2024-09-17T22:59:48.946 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941766+0000", 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 5.9959999999999999e-06 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "queued_for_pg", 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.941771+0000", 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.887e-06 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "reached_pg", 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.943219+0000", 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.0014486480000000001 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.947 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "started", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.943323+0000", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00010408000000000001 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "waiting for subops from 4", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.943416+0000", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 9.2853e-05 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.944348+0000", 2024-09-17T22:59:48.948 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.00093192899999999996 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "op_commit", 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.945242+0000", 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 0.000894004 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "commit_sent", 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.945290+0000", 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 4.7694000000000001e-05 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: { 2024-09-17T22:59:48.949 INFO:teuthology.orchestra.run.smithi171.stdout: "event": "done", 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: "time": "2024-09-17T22:59:39.945329+0000", 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: "duration": 3.9489000000000001e-05 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-17T22:59:48.950 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-17T22:59:48.951 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:59:48.980 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T22:59:48.993 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:49.066 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:49.067 INFO:teuthology.orchestra.run.smithi159.stdout: "num_ops": 0 2024-09-17T22:59:49.067 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:49.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:59:49.112 INFO:teuthology.orchestra.run.smithi159.stdout:mon.a: {} 2024-09-17T22:59:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:59:49.122 INFO:teuthology.orchestra.run.smithi159.stdout:mon.b: {} 2024-09-17T22:59:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:59:49.130 INFO:teuthology.orchestra.run.smithi159.stdout:mon.c: {} 2024-09-17T22:59:49.131 INFO:teuthology.orchestra.run.smithi159.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:59:49.141 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T22:59:49.184 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:49.184 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [], 2024-09-17T22:59:49.185 INFO:teuthology.orchestra.run.smithi159.stdout: "complaint_time": 30, 2024-09-17T22:59:49.185 INFO:teuthology.orchestra.run.smithi159.stdout: "num_blocked_ops": 0 2024-09-17T22:59:49.185 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:49.192 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "size": 20, 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 600, 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "ops": [ 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26, version: 76'382, data_included: [0~631961,1179648~65536,4194304~131072], data_size: 828569, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:234ec886:::rbd_data.118827e1bedb.0000000000000004:26@76'382, size: 7340032, copy_subset: [(0, 7340032)], clone_subset: {}, snapset: 26=[]:{26: [26]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7340032, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.383758833999998, 2024-09-17T22:59:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021080690000000002, 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954949+0000", 2024-09-17T22:59:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.954950+0000", 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955691+0000", 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074070899999999999 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955710+0000", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9463999999999999e-05 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955731+0000", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0437000000000001e-05 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955812+0000", 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.1221e-05 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.955822+0000", 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0016e-05 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.957057+0000", 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0012356509999999999 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.354177563, 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001379094, 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.984531+0000", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3799999999999997e-07 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985075+0000", 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00054357699999999995 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985092+0000", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.7041000000000002e-05 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985103+0000", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0740999999999999e-05 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985162+0000", 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.9290999999999999e-05 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985170+0000", 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.7579999999999992e-06 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.985910+0000", 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00074014800000000004 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30, version: 86'404, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30@86'404, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 30=[]:{30: [30]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.351835809999997, 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013103189999999999, 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986872+0000", 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.986873+0000", 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987326+0000", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00045334100000000001 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987343+0000", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6495e-05 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987354+0000", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0791999999999999e-05 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987411+0000", 2024-09-17T22:59:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7445000000000003e-05 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.987419+0000", 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0139999999999998e-06 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:10.988183+0000", 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00076374600000000004 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.4 124/123 [PushOp(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35, version: 91'439, data_included: [12288~28672,102400~28672,143360~28672,233472~28672,274432~28672,364544~28672,405504~28672,495616~28672,536576~28672,626688~28672,667648~28672,757760~28672,798720~28672,888832~28672,929792~20480,1019904~28672], data_size: 450560, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2775da4b:::rbd_data.1188584c80a2.0000000000000073:35@91'439, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 35=[]:{35: [35]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.335070031000001, 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017163370000000001, 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003638+0000", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003639+0000", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.71e-07 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003943+0000", 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00030434599999999999 2024-09-17T22:59:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003969+0000", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5700000000000001e-05 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.003994+0000", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4416000000000001e-05 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004076+0000", 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2137000000000003e-05 2024-09-17T22:59:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.004086+0000", 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0271999999999999e-05 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.005355+0000", 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001268595 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1528 3.4 e124/123 3:2363738a:::rbd_data.1188b9dbd70.0000000000000032:23 v 124'482, mlcod=115'479)", 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.284580407999997, 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022419990000000002, 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054128+0000", 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.6899999999999997e-07 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054135+0000", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2750000000000001e-06 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054146+0000", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0829000000000001e-05 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054153+0000", 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0099999999999998e-06 2024-09-17T22:59:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054191+0000", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8550999999999997e-05 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.054205+0000", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3891e-05 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056337+0000", 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021323190000000001 2024-09-17T22:59:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.056370+0000", 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.2455e-05 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1529 3.4 e124/123 3:220bdc83:::rbd_data.1188f4e2c696.000000000000010a:24 v 124'484, mlcod=115'480)", 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.281502174000003, 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0025955399999999999, 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057206+0000", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057207+0000", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.7899999999999998e-07 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057212+0000", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.0479999999999998e-06 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057221+0000", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.3449999999999995e-06 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057227+0000", 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3369999999999996e-06 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057277+0000", 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9292000000000002e-05 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057290+0000", 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3794999999999999e-05 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059759+0000", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0024690379999999998 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.059802+0000", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2106e-05 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1530 3.4 e124/123 3:222138c9:::rbd_data.1188f4e2c696.000000000000003e:24 v 124'486, mlcod=115'480)", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.281012701999998, 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0023558860000000002, 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057696+0000", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057703+0000", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8859999999999998e-06 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057711+0000", 2024-09-17T22:59:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2949999999999994e-06 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057717+0000", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.4779999999999999e-06 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057755+0000", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.8606000000000003e-05 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.057767+0000", 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.1484999999999999e-05 2024-09-17T22:59:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060021+0000", 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0022539080000000002 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.060051+0000", 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0679000000000002e-05 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1534 3.4 e124/123 3:256e1dec:::rbd_data.118898d3e685.0000000000000002:30 v 124'494, mlcod=124'490)", 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 38.268347503000001, 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001260253, 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070361+0000", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.68e-07 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070367+0000", 2024-09-17T22:59:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.8780000000000003e-06 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070377+0000", 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.7440000000000002e-06 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070384+0000", 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9140000000000002e-06 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070425+0000", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0800999999999998e-05 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.070446+0000", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.1603e-05 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071578+0000", 2024-09-17T22:59:49.376 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001131878 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:11.071621+0000", 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.2766999999999999e-05 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "MOSDPGPush(3.11 125/124 [PushOp(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head, version: 48'115, data_included: [0~262144,524288~287322,819200~49152,942080~49152], data_size: 647770, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:8ac66e66:::rbd_data.1188887d44de.0000000000000004:head@48'115, size: 1356976, copy_subset: [(0, 1356976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1356976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:49.377 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 37.20387152, 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.002060559, 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "started", 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.378 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.134837+0000", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135765+0000", 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00092785999999999997 2024-09-17T22:59:49.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135788+0000", 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.2623e-05 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135808+0000", 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0049999999999999e-05 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.380 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135897+0000", 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.8865999999999999e-05 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.135909+0000", 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2482e-05 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:12.136897+0000", 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00098805099999999999 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1611 3.0 e139/126 3:0562d798:::rbd_data.1188457bc75a.000000000000000d:19 v 139'368, mlcod=130'365)", 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.461270653, 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0014449560000000001, 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.382 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877438+0000", 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.6800000000000002e-07 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.383 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877444+0000", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1809999999999998e-06 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877488+0000", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.3646000000000002e-05 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877497+0000", 2024-09-17T22:59:49.384 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.9919999999999996e-06 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877600+0000", 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00010299800000000001 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.877624+0000", 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4329e-05 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878837+0000", 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001212631 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878883+0000", 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.5911000000000001e-05 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1612 3.4 e139/123 3:268a8c58:::rbd_data.1188457bc75a.0000000000000001:19 v 139'535, mlcod=137'532)", 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:49.386 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 23.460497258, 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0021832689999999998, 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878211+0000", 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878212+0000", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.7599999999999996e-07 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878220+0000", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.9019999999999999e-06 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878240+0000", 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.984e-05 2024-09-17T22:59:49.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878248+0000", 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.0839999999999992e-06 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878316+0000", 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.8392999999999994e-05 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.389 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.878346+0000", 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9958999999999999e-05 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880345+0000", 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001999228 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:25.880394+0000", 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8986999999999997e-05 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1614 3.0 e140/126 3:01b2b794:::rbd_data.118894cf7479.0000000000000031:18 v 140'370, mlcod=137'366)", 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 22.466470244, 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0018540049999999999, 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872238+0000", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5199999999999998e-07 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872244+0000", 2024-09-17T22:59:49.392 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.558e-06 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872263+0000", 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9357000000000001e-05 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872272+0000", 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.1710000000000008e-06 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.393 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872343+0000", 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.0766999999999995e-05 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.872369+0000", 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.5870000000000001e-05 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874055+0000", 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001685576 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.394 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.874092+0000", 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.7453999999999997e-05 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1618 3.4 e140/123 3:277a93c4:::rbd_data.118894cf7479.0000000000000000:18 v 140'538, mlcod=139'535)", 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 22.462675199, 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013940280000000001, 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.395 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876033+0000", 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.396 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876034+0000", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9999999999999997e-07 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876040+0000", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2020000000000001e-06 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876049+0000", 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.799e-06 2024-09-17T22:59:49.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876055+0000", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1589999999999998e-06 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876103+0000", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7979999999999998e-05 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.876123+0000", 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9817e-05 2024-09-17T22:59:49.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877387+0000", 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001264041 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:26.877427+0000", 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.0330000000000002e-05 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:202353 3.4 e148/123 3:268e54a9:::rbd_object_map.1188cfc725d6:head v 148'556, mlcod=148'554)", 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:34.905852+0000", 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 14.432856229, 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0029540899999999999, 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905852+0000", 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.400 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905852+0000", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905853+0000", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.4e-07 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905859+0000", 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2400000000000004e-06 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.401 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905870+0000", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0516000000000001e-05 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905874+0000", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905918+0000", 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4073000000000003e-05 2024-09-17T22:59:49.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.905933+0000", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4589e-05 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.908758+0000", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0028248570000000001 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:34.908806+0000", 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8239999999999999e-05 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.403 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1653 3.4 e152/123 3:260cd105:::rbd_data.11889f90e108.0000000000000009:c v 152'563, mlcod=148'560)", 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:38.942718+0000", 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.395990124000001, 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0015751090000000001, 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.404 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942718+0000", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942718+0000", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942719+0000", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.405 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942727+0000", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.6070000000000008e-06 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942736+0000", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 9.2529999999999993e-06 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942752+0000", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.5818e-05 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942900+0000", 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00014749899999999999 2024-09-17T22:59:49.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.942919+0000", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.9576e-05 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.944244+0000", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0013250899999999999 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:38.944293+0000", 2024-09-17T22:59:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.8736000000000002e-05 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:203425 3.4 e152/123 3:25de2026:::rbd_header.118834c4f370:head v 152'566, mlcod=152'564)", 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:39.075987+0000", 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.262720963, 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0017489249999999999, 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.408 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.075987+0000", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.075987+0000", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.075988+0000", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0330000000000001e-06 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.409 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.075995+0000", 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.2859999999999996e-06 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.076008+0000", 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3414000000000001e-05 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.076020+0000", 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.2255999999999999e-05 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.410 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.076081+0000", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1234000000000004e-05 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.076106+0000", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.4284e-05 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.077691+0000", 2024-09-17T22:59:49.411 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00158566 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.077736+0000", 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.4758000000000001e-05 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:203429 3.4 e152/123 3:25de2026:::rbd_header.118834c4f370:head v 152'567, mlcod=152'565)", 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:39.079185+0000", 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 10.259523371, 2024-09-17T22:59:49.412 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001270918, 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079185+0000", 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079185+0000", 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079186+0000", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.342e-06 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079193+0000", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9149999999999999e-06 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079207+0000", 2024-09-17T22:59:49.414 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4238e-05 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079215+0000", 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6059999999999997e-06 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079265+0000", 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.9975000000000002e-05 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.415 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.079286+0000", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.0866999999999999e-05 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.080425+0000", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001139234 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.080456+0000", 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.0741000000000003e-05 2024-09-17T22:59:49.416 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(osd.1.0:1661 3.0 e153/126 3:04763332:::rbd_data.118834c4f370.000000000000004e:b v 153'385, mlcod=147'382)", 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:39.866843+0000", 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.471865008, 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.001260473, 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.417 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866843+0000", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866843+0000", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866844+0000", 2024-09-17T22:59:49.418 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.9999999999999999e-07 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866850+0000", 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.1519999999999997e-06 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866864+0000", 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4184e-05 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.419 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866878+0000", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.4088e-05 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866945+0000", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.7219999999999997e-05 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.866968+0000", 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.3306999999999999e-05 2024-09-17T22:59:49.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868064+0000", 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00109588 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868104+0000", 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9341999999999997e-05 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.421 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:203450 3.4 e153/123 3:25de2026:::rbd_header.118834c4f370:head v 153'572, mlcod=152'568)", 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:39.868911+0000", 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.4697970750000007, 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0031481209999999998, 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868911+0000", 2024-09-17T22:59:49.422 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868911+0000", 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868912+0000", 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.82e-07 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.423 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868917+0000", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.2220000000000001e-06 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868925+0000", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 8.2060000000000007e-06 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868931+0000", 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 5.913e-06 2024-09-17T22:59:49.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868971+0000", 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9879000000000002e-05 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.868984+0000", 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.3499999999999999e-05 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.425 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.872018+0000", 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0030334060000000002 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.872059+0000", 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 4.1412999999999998e-05 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "description": "osd_repop(client.4488.0:203473 3.4 e153/123 3:25de2026:::rbd_header.118834c4f370:head v 153'576, mlcod=153'574)", 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "initiated_at": "2024-09-17T22:59:39.897419+0000", 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "age": 9.4412889950000007, 2024-09-17T22:59:49.426 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.0019924529999999999, 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "continuous": false, 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "type_data": { 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "events": [ 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "initiated", 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897419+0000", 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "header_read", 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897419+0000", 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0 2024-09-17T22:59:49.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "throttled", 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897420+0000", 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 7.6499999999999998e-07 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "all_read", 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897426+0000", 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "dispatched", 2024-09-17T22:59:49.428 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897437+0000", 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.0988999999999999e-05 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "queued_for_pg", 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897444+0000", 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 6.9310000000000002e-06 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "reached_pg", 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897484+0000", 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 3.9524999999999999e-05 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.429 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "started", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.897501+0000", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 1.6969999999999998e-05 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "commit_sent", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.899384+0000", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 0.00188371 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "event": "done", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "time": "2024-09-17T22:59:39.899412+0000", 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: "duration": 2.7217999999999999e-05 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.430 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.431 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-17T22:59:49.431 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-17T22:59:49.431 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T22:59:49.866 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T22:59:49.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T22:59:51.010 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:59:51.014 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T22:59:51.029 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T22:59:52.008 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T22:59:52.015 INFO:teuthology.orchestra.run.smithi159.stderr:noscrub is unset 2024-09-17T22:59:52.027 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T22:59:52.033 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:59:53.023 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T22:59:53.024 INFO:teuthology.orchestra.run.smithi159.stderr:nodeep-scrub is unset 2024-09-17T22:59:53.042 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T22:59:54.025 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T22:59:54.041 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T22:59:54.873 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T22:59:54.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T22:59:56.050 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T22:59:56.067 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T22:59:57.059 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T22:59:57.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T22:59:58.062 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T22:59:58.078 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-17T22:59:59.072 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T22:59:59.088 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T22:59:59.876 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T22:59:59.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-17T23:00:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T23:00:01.105 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T23:00:02.091 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T23:00:02.107 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T23:00:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T23:00:03.120 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T23:00:04.106 INFO:teuthology.orchestra.run.smithi159.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T23:00:04.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T23:00:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T23:00:05.144 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T23:00:05.144 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T23:00:05.145 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T23:00:05.145 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T23:00:05.145 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T23:00:05.145 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:00:05.422 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:05.422 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":178,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T23:00:05.114221+0000","last_up_change":"2024-09-17T22:59:09.941537+0000","last_in_change":"2024-09-17T22:58:51.164342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:57:08.859324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:57:17.992936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":74,"up_from":123,"up_thru":130,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6817","nonce":3362624831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6819","nonce":3362624831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6823","nonce":3362624831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6821","nonce":3362624831}]},"public_addr":"172.21.15.159:6817/3362624831","cluster_addr":"172.21.15.159:6819/3362624831","heartbeat_back_addr":"172.21.15.159:6823/3362624831","heartbeat_front_addr":"172.21.15.159:6821/3362624831","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":71,"up_thru":126,"down_at":46,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6819","nonce":587621150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6821","nonce":587621150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6825","nonce":587621150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6823","nonce":587621150}]},"public_addr":"172.21.15.171:6819/587621150","cluster_addr":"172.21.15.171:6821/587621150","heartbeat_back_addr":"172.21.15.171:6825/587621150","heartbeat_front_addr":"172.21.15.171:6823/587621150","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":75},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":46},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[6,3]},{"pgid":"3.11","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:00:05.435 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T23:00:05.436 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:00:05.716 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:05.716 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":178,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T23:00:05.114221+0000","last_up_change":"2024-09-17T22:59:09.941537+0000","last_in_change":"2024-09-17T22:58:51.164342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:57:08.859324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:57:17.992936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":74,"up_from":123,"up_thru":130,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6817","nonce":3362624831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6819","nonce":3362624831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6823","nonce":3362624831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6821","nonce":3362624831}]},"public_addr":"172.21.15.159:6817/3362624831","cluster_addr":"172.21.15.159:6819/3362624831","heartbeat_back_addr":"172.21.15.159:6823/3362624831","heartbeat_front_addr":"172.21.15.159:6821/3362624831","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":71,"up_thru":126,"down_at":46,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6819","nonce":587621150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6821","nonce":587621150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6825","nonce":587621150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6823","nonce":587621150}]},"public_addr":"172.21.15.171:6819/587621150","cluster_addr":"172.21.15.171:6821/587621150","heartbeat_back_addr":"172.21.15.171:6825/587621150","heartbeat_front_addr":"172.21.15.171:6823/587621150","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":75},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":46},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[6,3]},{"pgid":"3.11","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:00:05.729 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T23:00:05.729 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T23:00:05.730 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T23:00:05.730 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T23:00:05.731 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T23:00:05.731 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T23:00:05.732 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T23:00:05.732 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T23:00:05.968 INFO:teuthology.orchestra.run.smithi159.stdout:304942678041 2024-09-17T23:00:05.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:00:05.969 INFO:teuthology.orchestra.run.smithi159.stdout:60129542186 2024-09-17T23:00:05.969 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:00:05.974 INFO:teuthology.orchestra.run.smithi159.stdout:528280977421 2024-09-17T23:00:05.975 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:00:05.976 INFO:teuthology.orchestra.run.smithi159.stdout:64424509481 2024-09-17T23:00:05.977 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T23:00:05.981 INFO:teuthology.orchestra.run.smithi159.stdout:55834574889 2024-09-17T23:00:05.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T23:00:05.982 INFO:teuthology.orchestra.run.smithi159.stdout:60129542185 2024-09-17T23:00:05.983 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:00:05.983 INFO:teuthology.orchestra.run.smithi159.stdout:60129542185 2024-09-17T23:00:05.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:00:05.984 INFO:teuthology.orchestra.run.smithi159.stdout:64424509481 2024-09-17T23:00:05.985 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:00:06.518 INFO:teuthology.orchestra.run.smithi159.stdout:60129542185 2024-09-17T23:00:06.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.0 2024-09-17T23:00:06.548 INFO:teuthology.orchestra.run.smithi159.stdout:304942678040 2024-09-17T23:00:06.548 INFO:teuthology.orchestra.run.smithi159.stdout:528280977420 2024-09-17T23:00:06.566 INFO:teuthology.orchestra.run.smithi159.stdout:64424509480 2024-09-17T23:00:06.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 304942678041 got 304942678040 for osd.5 2024-09-17T23:00:06.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 528280977421 got 528280977420 for osd.2 2024-09-17T23:00:06.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509481 got 64424509480 for osd.6 2024-09-17T23:00:06.614 INFO:teuthology.orchestra.run.smithi159.stdout:55834574888 2024-09-17T23:00:06.620 INFO:teuthology.orchestra.run.smithi159.stdout:60129542184 2024-09-17T23:00:06.623 INFO:teuthology.orchestra.run.smithi159.stdout:64424509480 2024-09-17T23:00:06.627 INFO:teuthology.orchestra.run.smithi159.stdout:60129542184 2024-09-17T23:00:06.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574888 for osd.7 2024-09-17T23:00:06.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.3 2024-09-17T23:00:06.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509481 got 64424509480 for osd.4 2024-09-17T23:00:06.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.1 2024-09-17T23:00:07.539 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:00:07.570 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:00:07.571 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:00:07.581 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T23:00:07.632 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T23:00:07.637 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:00:07.641 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:00:07.644 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:00:08.053 INFO:teuthology.orchestra.run.smithi159.stdout:60129542186 2024-09-17T23:00:08.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.0 2024-09-17T23:00:08.073 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.096 INFO:teuthology.orchestra.run.smithi159.stdout:528280977421 2024-09-17T23:00:08.102 INFO:teuthology.orchestra.run.smithi159.stdout:64424509481 2024-09-17T23:00:08.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 528280977421 got 528280977421 for osd.2 2024-09-17T23:00:08.112 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509481 got 64424509481 for osd.6 2024-09-17T23:00:08.121 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.132 INFO:teuthology.orchestra.run.smithi159.stdout:304942678041 2024-09-17T23:00:08.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 304942678041 got 304942678041 for osd.5 2024-09-17T23:00:08.148 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.164 INFO:teuthology.orchestra.run.smithi159.stdout:55834574889 2024-09-17T23:00:08.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574889 for osd.7 2024-09-17T23:00:08.177 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.179 INFO:teuthology.orchestra.run.smithi159.stdout:60129542185 2024-09-17T23:00:08.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542185 for osd.1 2024-09-17T23:00:08.192 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.229 INFO:teuthology.orchestra.run.smithi159.stdout:64424509481 2024-09-17T23:00:08.233 INFO:teuthology.orchestra.run.smithi159.stdout:60129542185 2024-09-17T23:00:08.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509481 got 64424509481 for osd.4 2024-09-17T23:00:08.243 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542185 for osd.3 2024-09-17T23:00:08.247 DEBUG:teuthology.parallel:result is None 2024-09-17T23:00:08.247 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T23:00:08.247 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:08.564 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:08.575 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T23:00:08.222986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21144,"ondisk_log_size":21144,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420480,"kb_used_data":17856,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554240,"statfs":{"total":773094113280,"available":772663541760,"internally_reserved":0,"allocated":18284544,"data_stored":9507366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001198"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4933,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014715+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014715+0000","last_peered":"2024-09-17T23:00:06.014715+0000","last_clean":"2024-09-17T23:00:06.014715+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014715+0000","last_undegraded":"2024-09-17T23:00:06.014715+0000","last_fullsized":"2024-09-17T23:00:06.014715+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:11.778320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15262,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014587+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014587+0000","last_peered":"2024-09-17T23:00:06.014587+0000","last_clean":"2024-09-17T23:00:06.014587+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014587+0000","last_undegraded":"2024-09-17T23:00:06.014587+0000","last_fullsized":"2024-09-17T23:00:06.014587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:55.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10768,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107495+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:04.107495+0000","last_peered":"2024-09-17T23:00:04.107495+0000","last_clean":"2024-09-17T23:00:04.107495+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:04.107495+0000","last_undegraded":"2024-09-17T23:00:04.107495+0000","last_fullsized":"2024-09-17T23:00:04.107495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014747+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014747+0000","last_peered":"2024-09-17T23:00:06.014747+0000","last_clean":"2024-09-17T23:00:06.014747+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014747+0000","last_undegraded":"2024-09-17T23:00:06.014747+0000","last_fullsized":"2024-09-17T23:00:06.014747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:16.563222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3047,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107640+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:04.107640+0000","last_peered":"2024-09-17T23:00:04.107640+0000","last_clean":"2024-09-17T23:00:04.107640+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:04.107640+0000","last_undegraded":"2024-09-17T23:00:04.107640+0000","last_fullsized":"2024-09-17T23:00:04.107640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":344,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071129+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:05.071129+0000","last_peered":"2024-09-17T23:00:05.071129+0000","last_clean":"2024-09-17T23:00:05.071129+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:05.071129+0000","last_undegraded":"2024-09-17T23:00:05.071129+0000","last_fullsized":"2024-09-17T23:00:05.071129+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10497,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014637+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014637+0000","last_peered":"2024-09-17T23:00:06.014637+0000","last_clean":"2024-09-17T23:00:06.014637+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014637+0000","last_undegraded":"2024-09-17T23:00:06.014637+0000","last_fullsized":"2024-09-17T23:00:06.014637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:27.063226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5508,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107780+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:04.107780+0000","last_peered":"2024-09-17T23:00:04.107780+0000","last_clean":"2024-09-17T23:00:04.107780+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:04.107780+0000","last_undegraded":"2024-09-17T23:00:04.107780+0000","last_fullsized":"2024-09-17T23:00:04.107780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.135826+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T23:00:05.135826+0000","last_peered":"2024-09-17T23:00:05.135826+0000","last_clean":"2024-09-17T23:00:05.135826+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:05.135826+0000","last_undegraded":"2024-09-17T23:00:05.135826+0000","last_fullsized":"2024-09-17T23:00:05.135826+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13351,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014449+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014449+0000","last_peered":"2024-09-17T23:00:06.014449+0000","last_clean":"2024-09-17T23:00:06.014449+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014449+0000","last_undegraded":"2024-09-17T23:00:06.014449+0000","last_fullsized":"2024-09-17T23:00:06.014449+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:14.711737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4341,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3687,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071104+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:05.071104+0000","last_peered":"2024-09-17T23:00:05.071104+0000","last_clean":"2024-09-17T23:00:05.071104+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:05.071104+0000","last_undegraded":"2024-09-17T23:00:05.071104+0000","last_fullsized":"2024-09-17T23:00:05.071104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5834,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"160'805","reported_seq":13573,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014474+0000","last_change":"2024-09-17T22:59:47.991028+0000","last_active":"2024-09-17T23:00:06.014474+0000","last_peered":"2024-09-17T23:00:06.014474+0000","last_clean":"2024-09-17T23:00:06.014474+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:06.014474+0000","last_undegraded":"2024-09-17T23:00:06.014474+0000","last_fullsized":"2024-09-17T23:00:06.014474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:28.589908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21024,"ondisk_log_size":21024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45744,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326096,"statfs":{"total":96636764160,"available":96589922304,"internally_reserved":0,"allocated":2125824,"data_stored":1036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678042,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51320,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320520,"statfs":{"total":96636764160,"available":96584212480,"internally_reserved":0,"allocated":2129920,"data_stored":1036435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977421,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34692,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337148,"statfs":{"total":96636764160,"available":96601239552,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:08.577 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:08.855 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:08.856 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:08.867 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T23:00:08.222986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21144,"ondisk_log_size":21144,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420480,"kb_used_data":17856,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554240,"statfs":{"total":773094113280,"available":772663541760,"internally_reserved":0,"allocated":18284544,"data_stored":9507366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001198"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4933,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014715+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014715+0000","last_peered":"2024-09-17T23:00:06.014715+0000","last_clean":"2024-09-17T23:00:06.014715+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014715+0000","last_undegraded":"2024-09-17T23:00:06.014715+0000","last_fullsized":"2024-09-17T23:00:06.014715+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:11.778320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15262,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014587+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014587+0000","last_peered":"2024-09-17T23:00:06.014587+0000","last_clean":"2024-09-17T23:00:06.014587+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014587+0000","last_undegraded":"2024-09-17T23:00:06.014587+0000","last_fullsized":"2024-09-17T23:00:06.014587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:55.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10768,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107495+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:04.107495+0000","last_peered":"2024-09-17T23:00:04.107495+0000","last_clean":"2024-09-17T23:00:04.107495+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:04.107495+0000","last_undegraded":"2024-09-17T23:00:04.107495+0000","last_fullsized":"2024-09-17T23:00:04.107495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014747+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014747+0000","last_peered":"2024-09-17T23:00:06.014747+0000","last_clean":"2024-09-17T23:00:06.014747+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014747+0000","last_undegraded":"2024-09-17T23:00:06.014747+0000","last_fullsized":"2024-09-17T23:00:06.014747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:16.563222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3047,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107640+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:04.107640+0000","last_peered":"2024-09-17T23:00:04.107640+0000","last_clean":"2024-09-17T23:00:04.107640+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:04.107640+0000","last_undegraded":"2024-09-17T23:00:04.107640+0000","last_fullsized":"2024-09-17T23:00:04.107640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":344,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071129+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:05.071129+0000","last_peered":"2024-09-17T23:00:05.071129+0000","last_clean":"2024-09-17T23:00:05.071129+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:05.071129+0000","last_undegraded":"2024-09-17T23:00:05.071129+0000","last_fullsized":"2024-09-17T23:00:05.071129+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10497,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014637+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014637+0000","last_peered":"2024-09-17T23:00:06.014637+0000","last_clean":"2024-09-17T23:00:06.014637+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014637+0000","last_undegraded":"2024-09-17T23:00:06.014637+0000","last_fullsized":"2024-09-17T23:00:06.014637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:27.063226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5508,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107780+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:04.107780+0000","last_peered":"2024-09-17T23:00:04.107780+0000","last_clean":"2024-09-17T23:00:04.107780+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:04.107780+0000","last_undegraded":"2024-09-17T23:00:04.107780+0000","last_fullsized":"2024-09-17T23:00:04.107780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.135826+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T23:00:05.135826+0000","last_peered":"2024-09-17T23:00:05.135826+0000","last_clean":"2024-09-17T23:00:05.135826+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:05.135826+0000","last_undegraded":"2024-09-17T23:00:05.135826+0000","last_fullsized":"2024-09-17T23:00:05.135826+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13351,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014449+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014449+0000","last_peered":"2024-09-17T23:00:06.014449+0000","last_clean":"2024-09-17T23:00:06.014449+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014449+0000","last_undegraded":"2024-09-17T23:00:06.014449+0000","last_fullsized":"2024-09-17T23:00:06.014449+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:14.711737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4341,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3687,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071104+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:05.071104+0000","last_peered":"2024-09-17T23:00:05.071104+0000","last_clean":"2024-09-17T23:00:05.071104+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:05.071104+0000","last_undegraded":"2024-09-17T23:00:05.071104+0000","last_fullsized":"2024-09-17T23:00:05.071104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5834,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"160'805","reported_seq":13573,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014474+0000","last_change":"2024-09-17T22:59:47.991028+0000","last_active":"2024-09-17T23:00:06.014474+0000","last_peered":"2024-09-17T23:00:06.014474+0000","last_clean":"2024-09-17T23:00:06.014474+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:06.014474+0000","last_undegraded":"2024-09-17T23:00:06.014474+0000","last_fullsized":"2024-09-17T23:00:06.014474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:28.589908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21024,"ondisk_log_size":21024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45744,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326096,"statfs":{"total":96636764160,"available":96589922304,"internally_reserved":0,"allocated":2125824,"data_stored":1036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678042,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51320,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320520,"statfs":{"total":96636764160,"available":96584212480,"internally_reserved":0,"allocated":2129920,"data_stored":1036435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977421,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34692,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337148,"statfs":{"total":96636764160,"available":96601239552,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:08.869 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T23:00:08.869 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T23:00:09.171 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T23:00:09.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T23:00:09.181 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:09.462 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:09.462 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:09.471 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T23:00:08.222986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21144,"ondisk_log_size":21144,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420480,"kb_used_data":17856,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554240,"statfs":{"total":773094113280,"available":772663541760,"internally_reserved":0,"allocated":18284544,"data_stored":9507366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001198"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4933,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014715+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014715+0000","last_peered":"2024-09-17T23:00:06.014715+0000","last_clean":"2024-09-17T23:00:06.014715+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014715+0000","last_undegraded":"2024-09-17T23:00:06.014715+0000","last_fullsized":"2024-09-17T23:00:06.014715+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:11.778320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15262,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014587+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014587+0000","last_peered":"2024-09-17T23:00:06.014587+0000","last_clean":"2024-09-17T23:00:06.014587+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014587+0000","last_undegraded":"2024-09-17T23:00:06.014587+0000","last_fullsized":"2024-09-17T23:00:06.014587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:55.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10768,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107495+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:04.107495+0000","last_peered":"2024-09-17T23:00:04.107495+0000","last_clean":"2024-09-17T23:00:04.107495+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:04.107495+0000","last_undegraded":"2024-09-17T23:00:04.107495+0000","last_fullsized":"2024-09-17T23:00:04.107495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014747+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014747+0000","last_peered":"2024-09-17T23:00:06.014747+0000","last_clean":"2024-09-17T23:00:06.014747+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014747+0000","last_undegraded":"2024-09-17T23:00:06.014747+0000","last_fullsized":"2024-09-17T23:00:06.014747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:16.563222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3047,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107640+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:04.107640+0000","last_peered":"2024-09-17T23:00:04.107640+0000","last_clean":"2024-09-17T23:00:04.107640+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:04.107640+0000","last_undegraded":"2024-09-17T23:00:04.107640+0000","last_fullsized":"2024-09-17T23:00:04.107640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":344,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071129+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:05.071129+0000","last_peered":"2024-09-17T23:00:05.071129+0000","last_clean":"2024-09-17T23:00:05.071129+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:05.071129+0000","last_undegraded":"2024-09-17T23:00:05.071129+0000","last_fullsized":"2024-09-17T23:00:05.071129+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10497,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014637+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014637+0000","last_peered":"2024-09-17T23:00:06.014637+0000","last_clean":"2024-09-17T23:00:06.014637+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014637+0000","last_undegraded":"2024-09-17T23:00:06.014637+0000","last_fullsized":"2024-09-17T23:00:06.014637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:27.063226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5508,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107780+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:04.107780+0000","last_peered":"2024-09-17T23:00:04.107780+0000","last_clean":"2024-09-17T23:00:04.107780+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:04.107780+0000","last_undegraded":"2024-09-17T23:00:04.107780+0000","last_fullsized":"2024-09-17T23:00:04.107780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.135826+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T23:00:05.135826+0000","last_peered":"2024-09-17T23:00:05.135826+0000","last_clean":"2024-09-17T23:00:05.135826+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:05.135826+0000","last_undegraded":"2024-09-17T23:00:05.135826+0000","last_fullsized":"2024-09-17T23:00:05.135826+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13351,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014449+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014449+0000","last_peered":"2024-09-17T23:00:06.014449+0000","last_clean":"2024-09-17T23:00:06.014449+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014449+0000","last_undegraded":"2024-09-17T23:00:06.014449+0000","last_fullsized":"2024-09-17T23:00:06.014449+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:14.711737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4341,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3687,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071104+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:05.071104+0000","last_peered":"2024-09-17T23:00:05.071104+0000","last_clean":"2024-09-17T23:00:05.071104+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:05.071104+0000","last_undegraded":"2024-09-17T23:00:05.071104+0000","last_fullsized":"2024-09-17T23:00:05.071104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5834,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"160'805","reported_seq":13573,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014474+0000","last_change":"2024-09-17T22:59:47.991028+0000","last_active":"2024-09-17T23:00:06.014474+0000","last_peered":"2024-09-17T23:00:06.014474+0000","last_clean":"2024-09-17T23:00:06.014474+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:06.014474+0000","last_undegraded":"2024-09-17T23:00:06.014474+0000","last_fullsized":"2024-09-17T23:00:06.014474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:28.589908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21024,"ondisk_log_size":21024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45744,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326096,"statfs":{"total":96636764160,"available":96589922304,"internally_reserved":0,"allocated":2125824,"data_stored":1036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678042,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51320,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320520,"statfs":{"total":96636764160,"available":96584212480,"internally_reserved":0,"allocated":2129920,"data_stored":1036435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977421,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34692,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337148,"statfs":{"total":96636764160,"available":96601239552,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:09.473 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:09.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T23:00:09.771+0000 7fef295f9640 0 --2- 172.21.15.159:0/1623544523 >> v2:172.21.15.159:3300/0 conn(0x7fef240b1bf0 0x7fef24142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T23:00:09.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:09.952 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:09.963 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T23:00:08.222986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21144,"ondisk_log_size":21144,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420480,"kb_used_data":17856,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554240,"statfs":{"total":773094113280,"available":772663541760,"internally_reserved":0,"allocated":18284544,"data_stored":9507366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001198"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4933,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014715+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014715+0000","last_peered":"2024-09-17T23:00:06.014715+0000","last_clean":"2024-09-17T23:00:06.014715+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014715+0000","last_undegraded":"2024-09-17T23:00:06.014715+0000","last_fullsized":"2024-09-17T23:00:06.014715+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:11.778320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15262,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014587+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014587+0000","last_peered":"2024-09-17T23:00:06.014587+0000","last_clean":"2024-09-17T23:00:06.014587+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014587+0000","last_undegraded":"2024-09-17T23:00:06.014587+0000","last_fullsized":"2024-09-17T23:00:06.014587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:55.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10768,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107495+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:04.107495+0000","last_peered":"2024-09-17T23:00:04.107495+0000","last_clean":"2024-09-17T23:00:04.107495+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:04.107495+0000","last_undegraded":"2024-09-17T23:00:04.107495+0000","last_fullsized":"2024-09-17T23:00:04.107495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014747+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014747+0000","last_peered":"2024-09-17T23:00:06.014747+0000","last_clean":"2024-09-17T23:00:06.014747+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014747+0000","last_undegraded":"2024-09-17T23:00:06.014747+0000","last_fullsized":"2024-09-17T23:00:06.014747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:16.563222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3047,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107640+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:04.107640+0000","last_peered":"2024-09-17T23:00:04.107640+0000","last_clean":"2024-09-17T23:00:04.107640+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:04.107640+0000","last_undegraded":"2024-09-17T23:00:04.107640+0000","last_fullsized":"2024-09-17T23:00:04.107640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":344,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071129+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:05.071129+0000","last_peered":"2024-09-17T23:00:05.071129+0000","last_clean":"2024-09-17T23:00:05.071129+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:05.071129+0000","last_undegraded":"2024-09-17T23:00:05.071129+0000","last_fullsized":"2024-09-17T23:00:05.071129+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10497,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014637+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014637+0000","last_peered":"2024-09-17T23:00:06.014637+0000","last_clean":"2024-09-17T23:00:06.014637+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014637+0000","last_undegraded":"2024-09-17T23:00:06.014637+0000","last_fullsized":"2024-09-17T23:00:06.014637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:27.063226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5508,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:04.107780+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:04.107780+0000","last_peered":"2024-09-17T23:00:04.107780+0000","last_clean":"2024-09-17T23:00:04.107780+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:04.107780+0000","last_undegraded":"2024-09-17T23:00:04.107780+0000","last_fullsized":"2024-09-17T23:00:04.107780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.135826+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T23:00:05.135826+0000","last_peered":"2024-09-17T23:00:05.135826+0000","last_clean":"2024-09-17T23:00:05.135826+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:05.135826+0000","last_undegraded":"2024-09-17T23:00:05.135826+0000","last_fullsized":"2024-09-17T23:00:05.135826+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13351,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014449+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014449+0000","last_peered":"2024-09-17T23:00:06.014449+0000","last_clean":"2024-09-17T23:00:06.014449+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014449+0000","last_undegraded":"2024-09-17T23:00:06.014449+0000","last_fullsized":"2024-09-17T23:00:06.014449+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:14.711737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4341,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3687,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071104+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:05.071104+0000","last_peered":"2024-09-17T23:00:05.071104+0000","last_clean":"2024-09-17T23:00:05.071104+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:05.071104+0000","last_undegraded":"2024-09-17T23:00:05.071104+0000","last_fullsized":"2024-09-17T23:00:05.071104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5834,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.071075+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:05.071075+0000","last_peered":"2024-09-17T23:00:05.071075+0000","last_clean":"2024-09-17T23:00:05.071075+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:05.071075+0000","last_undegraded":"2024-09-17T23:00:05.071075+0000","last_fullsized":"2024-09-17T23:00:05.071075+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"160'805","reported_seq":13573,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014474+0000","last_change":"2024-09-17T22:59:47.991028+0000","last_active":"2024-09-17T23:00:06.014474+0000","last_peered":"2024-09-17T23:00:06.014474+0000","last_clean":"2024-09-17T23:00:06.014474+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:06.014474+0000","last_undegraded":"2024-09-17T23:00:06.014474+0000","last_fullsized":"2024-09-17T23:00:06.014474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:28.589908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21024,"ondisk_log_size":21024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45744,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326096,"statfs":{"total":96636764160,"available":96589922304,"internally_reserved":0,"allocated":2125824,"data_stored":1036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678042,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51320,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320520,"statfs":{"total":96636764160,"available":96584212480,"internally_reserved":0,"allocated":2129920,"data_stored":1036435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977421,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34692,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337148,"statfs":{"total":96636764160,"available":96601239552,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:09.965 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T23:00:09.966 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:10.244 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:10.245 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:10.256 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-17T23:00:10.223501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21144,"ondisk_log_size":21144,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420504,"kb_used_data":17880,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554216,"statfs":{"total":773094113280,"available":772663517184,"internally_reserved":0,"allocated":18309120,"data_stored":9519108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.127687"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4933,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014715+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014715+0000","last_peered":"2024-09-17T23:00:06.014715+0000","last_clean":"2024-09-17T23:00:06.014715+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014715+0000","last_undegraded":"2024-09-17T23:00:06.014715+0000","last_fullsized":"2024-09-17T23:00:06.014715+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:11.778320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15262,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014587+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014587+0000","last_peered":"2024-09-17T23:00:06.014587+0000","last_clean":"2024-09-17T23:00:06.014587+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014587+0000","last_undegraded":"2024-09-17T23:00:06.014587+0000","last_fullsized":"2024-09-17T23:00:06.014587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:04:55.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10770,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014804+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:06.014804+0000","last_peered":"2024-09-17T23:00:06.014804+0000","last_clean":"2024-09-17T23:00:06.014804+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:06.014804+0000","last_undegraded":"2024-09-17T23:00:06.014804+0000","last_fullsized":"2024-09-17T23:00:06.014804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014747+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014747+0000","last_peered":"2024-09-17T23:00:06.014747+0000","last_clean":"2024-09-17T23:00:06.014747+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014747+0000","last_undegraded":"2024-09-17T23:00:06.014747+0000","last_fullsized":"2024-09-17T23:00:06.014747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:16.563222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3049,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015041+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:06.015041+0000","last_peered":"2024-09-17T23:00:06.015041+0000","last_clean":"2024-09-17T23:00:06.015041+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:06.015041+0000","last_undegraded":"2024-09-17T23:00:06.015041+0000","last_fullsized":"2024-09-17T23:00:06.015041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":346,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015169+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:06.015169+0000","last_peered":"2024-09-17T23:00:06.015169+0000","last_clean":"2024-09-17T23:00:06.015169+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:06.015169+0000","last_undegraded":"2024-09-17T23:00:06.015169+0000","last_fullsized":"2024-09-17T23:00:06.015169+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10497,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014637+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014637+0000","last_peered":"2024-09-17T23:00:06.014637+0000","last_clean":"2024-09-17T23:00:06.014637+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014637+0000","last_undegraded":"2024-09-17T23:00:06.014637+0000","last_fullsized":"2024-09-17T23:00:06.014637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:27.063226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5510,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014996+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:06.014996+0000","last_peered":"2024-09-17T23:00:06.014996+0000","last_clean":"2024-09-17T23:00:06.014996+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:06.014996+0000","last_undegraded":"2024-09-17T23:00:06.014996+0000","last_fullsized":"2024-09-17T23:00:06.014996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.135826+0000","last_change":"2024-09-17T22:59:10.964686+0000","last_active":"2024-09-17T23:00:05.135826+0000","last_peered":"2024-09-17T23:00:05.135826+0000","last_clean":"2024-09-17T23:00:05.135826+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:05.135826+0000","last_undegraded":"2024-09-17T23:00:05.135826+0000","last_fullsized":"2024-09-17T23:00:05.135826+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:22.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13351,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014449+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014449+0000","last_peered":"2024-09-17T23:00:06.014449+0000","last_clean":"2024-09-17T23:00:06.014449+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014449+0000","last_undegraded":"2024-09-17T23:00:06.014449+0000","last_fullsized":"2024-09-17T23:00:06.014449+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:14.711737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4343,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015059+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:06.015059+0000","last_peered":"2024-09-17T23:00:06.015059+0000","last_clean":"2024-09-17T23:00:06.015059+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:06.015059+0000","last_undegraded":"2024-09-17T23:00:06.015059+0000","last_fullsized":"2024-09-17T23:00:06.015059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3689,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015133+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:06.015133+0000","last_peered":"2024-09-17T23:00:06.015133+0000","last_clean":"2024-09-17T23:00:06.015133+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:06.015133+0000","last_undegraded":"2024-09-17T23:00:06.015133+0000","last_fullsized":"2024-09-17T23:00:06.015133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5836,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015137+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:06.015137+0000","last_peered":"2024-09-17T23:00:06.015137+0000","last_clean":"2024-09-17T23:00:06.015137+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:06.015137+0000","last_undegraded":"2024-09-17T23:00:06.015137+0000","last_fullsized":"2024-09-17T23:00:06.015137+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"160'805","reported_seq":13573,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014474+0000","last_change":"2024-09-17T22:59:47.991028+0000","last_active":"2024-09-17T23:00:06.014474+0000","last_peered":"2024-09-17T23:00:06.014474+0000","last_clean":"2024-09-17T23:00:06.014474+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:06.014474+0000","last_undegraded":"2024-09-17T23:00:06.014474+0000","last_fullsized":"2024-09-17T23:00:06.014474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:28.589908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21024,"ondisk_log_size":21024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678042,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51328,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320512,"statfs":{"total":96636764160,"available":96584204288,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977422,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34692,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337148,"statfs":{"total":96636764160,"available":96601239552,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:10.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:00:10.538 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:10.539 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":178,"fsid":"33081d94-d15e-4bd4-ba4c-30022eb1be7a","created":"2024-09-17T22:56:50.056303+0000","modified":"2024-09-17T23:00:05.114221+0000","last_up_change":"2024-09-17T22:59:09.941537+0000","last_in_change":"2024-09-17T22:58:51.164342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:57:02.221581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:57:08.859324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:57:17.992936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"892321c3-409f-40e6-bfe1-c87f464d030f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6809","nonce":2087020665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6811","nonce":2087020665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6815","nonce":2087020665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2087020665},{"type":"v1","addr":"172.21.15.159:6813","nonce":2087020665}]},"public_addr":"172.21.15.159:6809/2087020665","cluster_addr":"172.21.15.159:6811/2087020665","heartbeat_back_addr":"172.21.15.159:6815/2087020665","heartbeat_front_addr":"172.21.15.159:6813/2087020665","state":["exists","up"]},{"osd":1,"uuid":"cd777f15-a6ad-4f0c-820b-ef529dd9f008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6801","nonce":237398697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6803","nonce":237398697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6807","nonce":237398697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":237398697},{"type":"v1","addr":"172.21.15.159:6805","nonce":237398697}]},"public_addr":"172.21.15.159:6801/237398697","cluster_addr":"172.21.15.159:6803/237398697","heartbeat_back_addr":"172.21.15.159:6807/237398697","heartbeat_front_addr":"172.21.15.159:6805/237398697","state":["exists","up"]},{"osd":2,"uuid":"bd68e905-6e66-4735-9de8-66a99530420f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":74,"up_from":123,"up_thru":130,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6817","nonce":3362624831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6819","nonce":3362624831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6823","nonce":3362624831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3362624831},{"type":"v1","addr":"172.21.15.159:6821","nonce":3362624831}]},"public_addr":"172.21.15.159:6817/3362624831","cluster_addr":"172.21.15.159:6819/3362624831","heartbeat_back_addr":"172.21.15.159:6823/3362624831","heartbeat_front_addr":"172.21.15.159:6821/3362624831","state":["exists","up"]},{"osd":3,"uuid":"28b3b7ca-d7d7-4ed9-8dfe-113d3579209c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6825","nonce":2987856795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6827","nonce":2987856795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6831","nonce":2987856795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2987856795},{"type":"v1","addr":"172.21.15.159:6829","nonce":2987856795}]},"public_addr":"172.21.15.159:6825/2987856795","cluster_addr":"172.21.15.159:6827/2987856795","heartbeat_back_addr":"172.21.15.159:6831/2987856795","heartbeat_front_addr":"172.21.15.159:6829/2987856795","state":["exists","up"]},{"osd":4,"uuid":"d379c252-e66a-4d0e-af0a-81f6bda48c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6827","nonce":2123531187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6829","nonce":2123531187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6833","nonce":2123531187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2123531187},{"type":"v1","addr":"172.21.15.171:6831","nonce":2123531187}]},"public_addr":"172.21.15.171:6827/2123531187","cluster_addr":"172.21.15.171:6829/2123531187","heartbeat_back_addr":"172.21.15.171:6833/2123531187","heartbeat_front_addr":"172.21.15.171:6831/2123531187","state":["exists","up"]},{"osd":5,"uuid":"e78a100f-1ff3-4af6-abd8-a069030ec951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":71,"up_thru":126,"down_at":46,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6819","nonce":587621150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6821","nonce":587621150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6825","nonce":587621150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":587621150},{"type":"v1","addr":"172.21.15.171:6823","nonce":587621150}]},"public_addr":"172.21.15.171:6819/587621150","cluster_addr":"172.21.15.171:6821/587621150","heartbeat_back_addr":"172.21.15.171:6825/587621150","heartbeat_front_addr":"172.21.15.171:6823/587621150","state":["exists","up"]},{"osd":6,"uuid":"fc1af848-dfaa-4e40-a0af-8982c70f7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6811","nonce":1505991260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6813","nonce":1505991260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6817","nonce":1505991260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1505991260},{"type":"v1","addr":"172.21.15.171:6815","nonce":1505991260}]},"public_addr":"172.21.15.171:6811/1505991260","cluster_addr":"172.21.15.171:6813/1505991260","heartbeat_back_addr":"172.21.15.171:6817/1505991260","heartbeat_front_addr":"172.21.15.171:6815/1505991260","state":["exists","up"]},{"osd":7,"uuid":"ccd75a18-ded4-4f75-a09a-926da5362a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6801","nonce":585405482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6803","nonce":585405482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6807","nonce":585405482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":585405482},{"type":"v1","addr":"172.21.15.171:6805","nonce":585405482}]},"public_addr":"172.21.15.171:6801/585405482","cluster_addr":"172.21.15.171:6803/585405482","heartbeat_back_addr":"172.21.15.171:6807/585405482","heartbeat_front_addr":"172.21.15.171:6805/585405482","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.101775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.929708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.562720+0000","dead_epoch":75},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.569375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:57:00.002784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.775667+0000","dead_epoch":46},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:59.558418+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:56:58.840790+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[6,3]},{"pgid":"3.11","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:00:11.553 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T23:00:11.553 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:11.689 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:11.689 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:11.690 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:11.700 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T23:00:11.976 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T23:00:12.066 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T23:00:12.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:12.190 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:12.191 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:12.191 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:12.201 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T23:00:12.478 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T23:00:12.491 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T23:00:12.491 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:12.614 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:12.614 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:12.614 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:12.625 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T23:00:12.901 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T23:00:12.915 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T23:00:12.915 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:13.039 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:13.040 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:13.040 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:13.050 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T23:00:13.331 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T23:00:13.344 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T23:00:13.345 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:13.468 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:13.468 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:13.468 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:13.479 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T23:00:13.761 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T23:00:13.774 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T23:00:13.774 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:13.896 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:13.896 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:13.896 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:13.907 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T23:00:14.188 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T23:00:14.202 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T23:00:14.202 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:14.324 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:14.324 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:14.325 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:14.336 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T23:00:14.615 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T23:00:14.628 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T23:00:14.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:00:14.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-17T23:00:14.931+0000 7f3671829640 0 --2- 172.21.15.159:0/2772393264 >> [v2:172.21.15.159:3301/0,v1:172.21.15.159:6790/0] conn(0x7f366c0b0d20 0x7f366c0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T23:00:14.954 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-17T23:00:14.954 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:00:14.954 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-17T23:00:14.965 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T23:00:15.248 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T23:00:15.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:15.544 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:15.545 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:15.556 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-17T23:00:14.224382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21159,"ondisk_log_size":21159,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420492,"kb_used_data":17880,"kb_used_omap":41,"kb_used_meta":402390,"kb_avail":754554228,"statfs":{"total":773094113280,"available":772663529472,"internally_reserved":0,"allocated":18309120,"data_stored":9519108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":412047641},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002248"},"pg_stats":[{"pgid":"3.1f","version":"161'499","reported_seq":4934,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014761+0000","last_change":"2024-09-17T22:59:47.992980+0000","last_active":"2024-09-17T23:00:06.014761+0000","last_peered":"2024-09-17T23:00:06.014761+0000","last_clean":"2024-09-17T23:00:06.014761+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:06.014761+0000","last_undegraded":"2024-09-17T23:00:06.014761+0000","last_fullsized":"2024-09-17T23:00:06.014761+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'325","reported_seq":3499,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127147+0000","last_change":"2024-09-17T22:59:47.997799+0000","last_active":"2024-09-17T23:00:05.127147+0000","last_peered":"2024-09-17T23:00:05.127147+0000","last_clean":"2024-09-17T23:00:05.127147+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:05.127147+0000","last_undegraded":"2024-09-17T23:00:05.127147+0000","last_fullsized":"2024-09-17T23:00:05.127147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:36.949906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"159'590","reported_seq":10055,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015317+0000","last_change":"2024-09-17T22:59:47.995895+0000","last_active":"2024-09-17T23:00:06.015317+0000","last_peered":"2024-09-17T23:00:06.015317+0000","last_clean":"2024-09-17T23:00:06.015317+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:06.015317+0000","last_undegraded":"2024-09-17T23:00:06.015317+0000","last_fullsized":"2024-09-17T23:00:06.015317+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:46.468393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'639","reported_seq":6248,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014845+0000","last_change":"2024-09-17T22:59:47.995013+0000","last_active":"2024-09-17T23:00:06.014845+0000","last_peered":"2024-09-17T23:00:06.014845+0000","last_clean":"2024-09-17T23:00:06.014845+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:06.014845+0000","last_undegraded":"2024-09-17T23:00:06.014845+0000","last_fullsized":"2024-09-17T23:00:06.014845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:26.204703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"161'757","reported_seq":15263,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014636+0000","last_change":"2024-09-17T22:59:47.991904+0000","last_active":"2024-09-17T23:00:06.014636+0000","last_peered":"2024-09-17T23:00:06.014636+0000","last_clean":"2024-09-17T23:00:06.014636+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:06.014636+0000","last_undegraded":"2024-09-17T23:00:06.014636+0000","last_fullsized":"2024-09-17T23:00:06.014636+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'449","reported_seq":10770,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014804+0000","last_change":"2024-09-17T22:59:47.991747+0000","last_active":"2024-09-17T23:00:06.014804+0000","last_peered":"2024-09-17T23:00:06.014804+0000","last_clean":"2024-09-17T23:00:06.014804+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:06.014804+0000","last_undegraded":"2024-09-17T23:00:06.014804+0000","last_fullsized":"2024-09-17T23:00:06.014804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:35.625813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"159'782","reported_seq":5712,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127674+0000","last_change":"2024-09-17T22:59:47.991043+0000","last_active":"2024-09-17T23:00:05.127674+0000","last_peered":"2024-09-17T23:00:05.127674+0000","last_clean":"2024-09-17T23:00:05.127674+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:05.127674+0000","last_undegraded":"2024-09-17T23:00:05.127674+0000","last_fullsized":"2024-09-17T23:00:05.127674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:32.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'517","reported_seq":5980,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126995+0000","last_change":"2024-09-17T22:59:47.996563+0000","last_active":"2024-09-17T23:00:05.126995+0000","last_peered":"2024-09-17T23:00:05.126995+0000","last_clean":"2024-09-17T23:00:05.126995+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:05.126995+0000","last_undegraded":"2024-09-17T23:00:05.126995+0000","last_fullsized":"2024-09-17T23:00:05.126995+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:59:09.742118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"158'703","reported_seq":4793,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:59:47.993176+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"160'1333","reported_seq":9792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014917+0000","last_change":"2024-09-17T22:59:47.995616+0000","last_active":"2024-09-17T23:00:06.014917+0000","last_peered":"2024-09-17T23:00:06.014917+0000","last_clean":"2024-09-17T23:00:06.014917+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:06.014917+0000","last_undegraded":"2024-09-17T23:00:06.014917+0000","last_fullsized":"2024-09-17T23:00:06.014917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:51.446662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"160'485","reported_seq":5027,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125716+0000","last_change":"2024-09-17T23:00:02.095606+0000","last_active":"2024-09-17T23:00:05.125716+0000","last_peered":"2024-09-17T23:00:05.125716+0000","last_clean":"2024-09-17T23:00:05.125716+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:05.125716+0000","last_undegraded":"2024-09-17T23:00:05.125716+0000","last_fullsized":"2024-09-17T23:00:05.125716+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:48.059465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'509","reported_seq":3049,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015041+0000","last_change":"2024-09-17T22:59:47.991745+0000","last_active":"2024-09-17T23:00:06.015041+0000","last_peered":"2024-09-17T23:00:06.015041+0000","last_clean":"2024-09-17T23:00:06.015041+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:06.015041+0000","last_undegraded":"2024-09-17T23:00:06.015041+0000","last_fullsized":"2024-09-17T23:00:06.015041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":346,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015169+0000","last_change":"2024-09-17T22:59:21.869903+0000","last_active":"2024-09-17T23:00:06.015169+0000","last_peered":"2024-09-17T23:00:06.015169+0000","last_clean":"2024-09-17T23:00:06.015169+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:06.015169+0000","last_undegraded":"2024-09-17T23:00:06.015169+0000","last_fullsized":"2024-09-17T23:00:06.015169+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:30.826616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"161'695","reported_seq":10498,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014681+0000","last_change":"2024-09-17T22:59:47.991880+0000","last_active":"2024-09-17T23:00:06.014681+0000","last_peered":"2024-09-17T23:00:06.014681+0000","last_clean":"2024-09-17T23:00:06.014681+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:06.014681+0000","last_undegraded":"2024-09-17T23:00:06.014681+0000","last_fullsized":"2024-09-17T23:00:06.014681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127812+0000","last_change":"2024-09-17T22:57:11.901349+0000","last_active":"2024-09-17T23:00:05.127812+0000","last_peered":"2024-09-17T23:00:05.127812+0000","last_clean":"2024-09-17T23:00:05.127812+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:05.127812+0000","last_undegraded":"2024-09-17T23:00:05.127812+0000","last_fullsized":"2024-09-17T23:00:05.127812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:15.171230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'533","reported_seq":9927,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127333+0000","last_change":"2024-09-17T22:59:47.997707+0000","last_active":"2024-09-17T23:00:05.127333+0000","last_peered":"2024-09-17T23:00:05.127333+0000","last_clean":"2024-09-17T23:00:05.127333+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:05.127333+0000","last_undegraded":"2024-09-17T23:00:05.127333+0000","last_fullsized":"2024-09-17T23:00:05.127333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:27.767086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125762+0000","last_change":"2024-09-17T22:57:12.701170+0000","last_active":"2024-09-17T23:00:05.125762+0000","last_peered":"2024-09-17T23:00:05.125762+0000","last_clean":"2024-09-17T23:00:05.125762+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:05.125762+0000","last_undegraded":"2024-09-17T23:00:05.125762+0000","last_fullsized":"2024-09-17T23:00:05.125762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:02:42.654823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"155'596","reported_seq":10095,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127722+0000","last_change":"2024-09-17T22:59:47.991205+0000","last_active":"2024-09-17T23:00:05.127722+0000","last_peered":"2024-09-17T23:00:05.127722+0000","last_clean":"2024-09-17T23:00:05.127722+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:05.127722+0000","last_undegraded":"2024-09-17T23:00:05.127722+0000","last_fullsized":"2024-09-17T23:00:05.127722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:59.916360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127764+0000","last_change":"2024-09-17T22:57:11.901338+0000","last_active":"2024-09-17T23:00:05.127764+0000","last_peered":"2024-09-17T23:00:05.127764+0000","last_clean":"2024-09-17T23:00:05.127764+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:05.127764+0000","last_undegraded":"2024-09-17T23:00:05.127764+0000","last_fullsized":"2024-09-17T23:00:05.127764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:35.974325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'780","reported_seq":4260,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015213+0000","last_change":"2024-09-17T22:59:47.994665+0000","last_active":"2024-09-17T23:00:06.015213+0000","last_peered":"2024-09-17T23:00:06.015213+0000","last_clean":"2024-09-17T23:00:06.015213+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:06.015213+0000","last_undegraded":"2024-09-17T23:00:06.015213+0000","last_fullsized":"2024-09-17T23:00:06.015213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:33.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'753","reported_seq":5510,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014996+0000","last_change":"2024-09-17T22:59:47.992278+0000","last_active":"2024-09-17T23:00:06.014996+0000","last_peered":"2024-09-17T23:00:06.014996+0000","last_clean":"2024-09-17T23:00:06.014996+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:06.014996+0000","last_undegraded":"2024-09-17T23:00:06.014996+0000","last_fullsized":"2024-09-17T23:00:06.014996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.619159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014810+0000","last_change":"2024-09-17T22:58:52.183539+0000","last_active":"2024-09-17T23:00:06.014810+0000","last_peered":"2024-09-17T23:00:06.014810+0000","last_clean":"2024-09-17T23:00:06.014810+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:06.014810+0000","last_undegraded":"2024-09-17T23:00:06.014810+0000","last_fullsized":"2024-09-17T23:00:06.014810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:56.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'410","reported_seq":19319,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127624+0000","last_change":"2024-09-17T22:59:47.991721+0000","last_active":"2024-09-17T23:00:05.127624+0000","last_peered":"2024-09-17T23:00:05.127624+0000","last_clean":"2024-09-17T23:00:05.127624+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:05.127624+0000","last_undegraded":"2024-09-17T23:00:05.127624+0000","last_fullsized":"2024-09-17T23:00:05.127624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:23.093094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:12.946425+0000","last_change":"2024-09-17T23:00:12.946462+0000","last_active":"2024-09-17T23:00:12.946425+0000","last_peered":"2024-09-17T23:00:12.946425+0000","last_clean":"2024-09-17T23:00:12.946425+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:12.946425+0000","last_undegraded":"2024-09-17T23:00:12.946425+0000","last_fullsized":"2024-09-17T23:00:12.946425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:12.946338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:12.946338+0000","last_clean_scrub_stamp":"2024-09-17T23:00:12.946338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:18.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'1263","reported_seq":13352,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014513+0000","last_change":"2024-09-17T22:59:47.991025+0000","last_active":"2024-09-17T23:00:06.014513+0000","last_peered":"2024-09-17T23:00:06.014513+0000","last_clean":"2024-09-17T23:00:06.014513+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:06.014513+0000","last_undegraded":"2024-09-17T23:00:06.014513+0000","last_fullsized":"2024-09-17T23:00:06.014513+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125542+0000","last_change":"2024-09-17T23:00:02.094119+0000","last_active":"2024-09-17T23:00:05.125542+0000","last_peered":"2024-09-17T23:00:05.125542+0000","last_clean":"2024-09-17T23:00:05.125542+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:05.125542+0000","last_undegraded":"2024-09-17T23:00:05.125542+0000","last_fullsized":"2024-09-17T23:00:05.125542+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:41.345928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"160'860","reported_seq":6326,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127089+0000","last_change":"2024-09-17T22:59:47.997221+0000","last_active":"2024-09-17T23:00:05.127089+0000","last_peered":"2024-09-17T23:00:05.127089+0000","last_clean":"2024-09-17T23:00:05.127089+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:05.127089+0000","last_undegraded":"2024-09-17T23:00:05.127089+0000","last_fullsized":"2024-09-17T23:00:05.127089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:56.630917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014786+0000","last_change":"2024-09-17T22:59:11.956429+0000","last_active":"2024-09-17T23:00:06.014786+0000","last_peered":"2024-09-17T23:00:06.014786+0000","last_clean":"2024-09-17T23:00:06.014786+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:06.014786+0000","last_undegraded":"2024-09-17T23:00:06.014786+0000","last_fullsized":"2024-09-17T23:00:06.014786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:08.867640+0000","last_clean_scrub_stamp":"2024-09-17T22:57:08.867640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:37.378305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125551+0000","last_change":"2024-09-17T23:00:02.094983+0000","last_active":"2024-09-17T23:00:05.125551+0000","last_peered":"2024-09-17T23:00:05.125551+0000","last_clean":"2024-09-17T23:00:05.125551+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:05.125551+0000","last_undegraded":"2024-09-17T23:00:05.125551+0000","last_fullsized":"2024-09-17T23:00:05.125551+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:02.843283+0000","last_clean_scrub_stamp":"2024-09-17T22:57:02.843283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:04.516933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"160'817","reported_seq":4005,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.126987+0000","last_change":"2024-09-17T22:59:47.996180+0000","last_active":"2024-09-17T23:00:05.126987+0000","last_peered":"2024-09-17T23:00:05.126987+0000","last_clean":"2024-09-17T23:00:05.126987+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:05.126987+0000","last_undegraded":"2024-09-17T23:00:05.126987+0000","last_fullsized":"2024-09-17T23:00:05.126987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:41.664597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'562","reported_seq":4343,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015059+0000","last_change":"2024-09-17T22:59:48.411173+0000","last_active":"2024-09-17T23:00:06.015059+0000","last_peered":"2024-09-17T23:00:06.015059+0000","last_clean":"2024-09-17T23:00:06.015059+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:06.015059+0000","last_undegraded":"2024-09-17T23:00:06.015059+0000","last_fullsized":"2024-09-17T23:00:06.015059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:30.423774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"159'565","reported_seq":3689,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015133+0000","last_change":"2024-09-17T22:59:48.410966+0000","last_active":"2024-09-17T23:00:06.015133+0000","last_peered":"2024-09-17T23:00:06.015133+0000","last_clean":"2024-09-17T23:00:06.015133+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:06.015133+0000","last_undegraded":"2024-09-17T23:00:06.015133+0000","last_fullsized":"2024-09-17T23:00:06.015133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:58.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'560","reported_seq":6526,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.127125+0000","last_change":"2024-09-17T22:59:47.996662+0000","last_active":"2024-09-17T23:00:05.127125+0000","last_peered":"2024-09-17T23:00:05.127125+0000","last_clean":"2024-09-17T23:00:05.127125+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:05.127125+0000","last_undegraded":"2024-09-17T23:00:05.127125+0000","last_fullsized":"2024-09-17T23:00:05.127125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:47.036331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'428","reported_seq":5261,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014724+0000","last_change":"2024-09-17T22:59:47.994142+0000","last_active":"2024-09-17T23:00:06.014724+0000","last_peered":"2024-09-17T23:00:06.014724+0000","last_clean":"2024-09-17T23:00:06.014724+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:06.014724+0000","last_undegraded":"2024-09-17T23:00:06.014724+0000","last_fullsized":"2024-09-17T23:00:06.014724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:42.736230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"156'449","reported_seq":5118,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125640+0000","last_change":"2024-09-17T23:00:02.096790+0000","last_active":"2024-09-17T23:00:05.125640+0000","last_peered":"2024-09-17T23:00:05.125640+0000","last_clean":"2024-09-17T23:00:05.125640+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:05.125640+0000","last_undegraded":"2024-09-17T23:00:05.125640+0000","last_fullsized":"2024-09-17T23:00:05.125640+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:34.980401+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"158'352","reported_seq":2196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125733+0000","last_change":"2024-09-17T23:00:02.093875+0000","last_active":"2024-09-17T23:00:05.125733+0000","last_peered":"2024-09-17T23:00:05.125733+0000","last_clean":"2024-09-17T23:00:05.125733+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:05.125733+0000","last_undegraded":"2024-09-17T23:00:05.125733+0000","last_fullsized":"2024-09-17T23:00:05.125733+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:27.583201+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"155'1098","reported_seq":9314,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125812+0000","last_change":"2024-09-17T23:00:02.094264+0000","last_active":"2024-09-17T23:00:05.125812+0000","last_peered":"2024-09-17T23:00:05.125812+0000","last_clean":"2024-09-17T23:00:05.125812+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:05.125812+0000","last_undegraded":"2024-09-17T23:00:05.125812+0000","last_fullsized":"2024-09-17T23:00:05.125812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:07.110160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"160'704","reported_seq":5836,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.015137+0000","last_change":"2024-09-17T22:59:48.410892+0000","last_active":"2024-09-17T23:00:06.015137+0000","last_peered":"2024-09-17T23:00:06.015137+0000","last_clean":"2024-09-17T23:00:06.015137+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:06.015137+0000","last_undegraded":"2024-09-17T23:00:06.015137+0000","last_fullsized":"2024-09-17T23:00:06.015137+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:42.596854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"155'532","reported_seq":4734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:06.014686+0000","last_change":"2024-09-17T22:59:47.994391+0000","last_active":"2024-09-17T23:00:06.014686+0000","last_peered":"2024-09-17T23:00:06.014686+0000","last_clean":"2024-09-17T23:00:06.014686+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:06.014686+0000","last_undegraded":"2024-09-17T23:00:06.014686+0000","last_fullsized":"2024-09-17T23:00:06.014686+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:24.870349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"178'820","reported_seq":13597,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:12.067342+0000","last_change":"2024-09-17T23:00:12.067380+0000","last_active":"2024-09-17T23:00:12.067342+0000","last_peered":"2024-09-17T23:00:12.067342+0000","last_clean":"2024-09-17T23:00:12.067342+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:12.067342+0000","last_undegraded":"2024-09-17T23:00:12.067342+0000","last_fullsized":"2024-09-17T23:00:12.067342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'820","last_scrub_stamp":"2024-09-17T23:00:12.067262+0000","last_deep_scrub":"178'820","last_deep_scrub_stamp":"2024-09-17T23:00:12.067262+0000","last_clean_scrub_stamp":"2024-09-17T23:00:12.067262+0000","objects_scrubbed":15,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:10.115263+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"160'674","reported_seq":9249,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:05.125558+0000","last_change":"2024-09-17T22:59:48.228813+0000","last_active":"2024-09-17T23:00:05.125558+0000","last_peered":"2024-09-17T23:00:05.125558+0000","last_clean":"2024-09-17T23:00:05.125558+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:05.125558+0000","last_undegraded":"2024-09-17T23:00:05.125558+0000","last_fullsized":"2024-09-17T23:00:05.125558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:57:18.926847+0000","last_clean_scrub_stamp":"2024-09-17T22:57:18.926847+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:50.893335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21039,"ondisk_log_size":21039,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678043,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":45422,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":46512326},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51328,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320512,"statfs":{"total":96636764160,"available":96584204288,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59644,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312196,"statfs":{"total":96636764160,"available":96575688704,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977423,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34680,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337160,"statfs":{"total":96636764160,"available":96601251840,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55500,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316340,"statfs":{"total":96636764160,"available":96579932160,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65792,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306048,"statfs":{"total":96636764160,"available":96569393152,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.561 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.562 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T22:57:08.867640+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.563 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T22:57:02.843283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=2, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.564 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.565 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.565 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T22:57:18.926847+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=57, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=0, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:00:15.565 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T23:00:35.566 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:00:35.858 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-17T23:00:35.859 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-17T23:00:35.870 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-17T23:00:34.229814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":452,"num_object_clones":0,"num_object_copies":904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":452,"num_whiteouts":444,"num_read":262382,"num_read_kb":796657,"num_write":23613,"num_write_kb":368752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":119485760,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21593,"ondisk_log_size":21593,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391320,"kb_used_data":17880,"kb_used_omap":41,"kb_used_meta":373270,"kb_avail":754583400,"statfs":{"total":773094113280,"available":772693401600,"internally_reserved":0,"allocated":18309120,"data_stored":9519108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42727,"internal_metadata":382228761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003420"},"pg_stats":[{"pgid":"3.1f","version":"178'506","reported_seq":4949,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.016058+0000","last_change":"2024-09-17T23:00:16.016095+0000","last_active":"2024-09-17T23:00:16.016058+0000","last_peered":"2024-09-17T23:00:16.016058+0000","last_clean":"2024-09-17T23:00:16.016058+0000","last_became_active":"2024-09-17T22:59:19.033833+0000","last_became_peered":"2024-09-17T22:59:19.033833+0000","last_unstale":"2024-09-17T23:00:16.016058+0000","last_undegraded":"2024-09-17T23:00:16.016058+0000","last_fullsized":"2024-09-17T23:00:16.016058+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'506","last_scrub_stamp":"2024-09-17T23:00:16.015979+0000","last_deep_scrub":"178'506","last_deep_scrub_stamp":"2024-09-17T23:00:16.015979+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.015979+0000","objects_scrubbed":7,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:25.410704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0021573410000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4441,"num_read_kb":24011,"num_write":478,"num_write_kb":24369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":7614285,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"178'337","reported_seq":3520,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.487070+0000","last_change":"2024-09-17T23:00:14.487108+0000","last_active":"2024-09-17T23:00:14.487070+0000","last_peered":"2024-09-17T23:00:14.487070+0000","last_clean":"2024-09-17T23:00:14.487070+0000","last_became_active":"2024-09-17T22:57:19.854637+0000","last_became_peered":"2024-09-17T22:57:19.854637+0000","last_unstale":"2024-09-17T23:00:14.487070+0000","last_undegraded":"2024-09-17T23:00:14.487070+0000","last_fullsized":"2024-09-17T23:00:14.487070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'337","last_scrub_stamp":"2024-09-17T23:00:14.486991+0000","last_deep_scrub":"178'337","last_deep_scrub_stamp":"2024-09-17T23:00:14.486991+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.486991+0000","objects_scrubbed":12,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:18:38.157911+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0023787890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2657,"num_read_kb":17546,"num_write":284,"num_write_kb":8945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"178'604","reported_seq":10078,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.003023+0000","last_change":"2024-09-17T23:00:16.003089+0000","last_active":"2024-09-17T23:00:16.003023+0000","last_peered":"2024-09-17T23:00:16.003023+0000","last_clean":"2024-09-17T23:00:16.003023+0000","last_became_active":"2024-09-17T22:58:54.863342+0000","last_became_peered":"2024-09-17T22:58:54.863342+0000","last_unstale":"2024-09-17T23:00:16.003023+0000","last_undegraded":"2024-09-17T23:00:16.003023+0000","last_fullsized":"2024-09-17T23:00:16.003023+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'604","last_scrub_stamp":"2024-09-17T23:00:16.002908+0000","last_deep_scrub":"178'604","last_deep_scrub_stamp":"2024-09-17T23:00:16.002908+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.002908+0000","objects_scrubbed":14,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:17.636930+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.002199229,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10490,"num_read_kb":31580,"num_write":596,"num_write_kb":6665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4909770,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'650","reported_seq":6268,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.029813+0000","last_change":"2024-09-17T23:00:17.029881+0000","last_active":"2024-09-17T23:00:17.029813+0000","last_peered":"2024-09-17T23:00:17.029813+0000","last_clean":"2024-09-17T23:00:17.029813+0000","last_became_active":"2024-09-17T22:58:54.862559+0000","last_became_peered":"2024-09-17T22:58:54.862559+0000","last_unstale":"2024-09-17T23:00:17.029813+0000","last_undegraded":"2024-09-17T23:00:17.029813+0000","last_fullsized":"2024-09-17T23:00:17.029813+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'650","last_scrub_stamp":"2024-09-17T23:00:17.029691+0000","last_deep_scrub":"178'650","last_deep_scrub_stamp":"2024-09-17T23:00:17.029691+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.029691+0000","objects_scrubbed":12,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:22.197947+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00149383,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6887,"num_read_kb":17420,"num_write":667,"num_write_kb":6189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":4368818,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"178'767","reported_seq":15281,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.994785+0000","last_change":"2024-09-17T23:00:14.994817+0000","last_active":"2024-09-17T23:00:14.994785+0000","last_peered":"2024-09-17T23:00:14.994785+0000","last_clean":"2024-09-17T23:00:14.994785+0000","last_became_active":"2024-09-17T22:59:19.031831+0000","last_became_peered":"2024-09-17T22:59:19.031831+0000","last_unstale":"2024-09-17T23:00:14.994785+0000","last_undegraded":"2024-09-17T23:00:14.994785+0000","last_fullsized":"2024-09-17T23:00:14.994785+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'767","last_scrub_stamp":"2024-09-17T23:00:14.994709+0000","last_deep_scrub":"178'767","last_deep_scrub_stamp":"2024-09-17T23:00:14.994709+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.994709+0000","objects_scrubbed":10,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:49.522253+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001268019,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":16844,"num_read_kb":56851,"num_write":759,"num_write_kb":17001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":17156254,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'462","reported_seq":10792,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.366919+0000","last_change":"2024-09-17T23:00:16.366993+0000","last_active":"2024-09-17T23:00:16.366919+0000","last_peered":"2024-09-17T23:00:16.366919+0000","last_clean":"2024-09-17T23:00:16.366919+0000","last_became_active":"2024-09-17T22:57:19.852399+0000","last_became_peered":"2024-09-17T22:57:19.852399+0000","last_unstale":"2024-09-17T23:00:16.366919+0000","last_undegraded":"2024-09-17T23:00:16.366919+0000","last_fullsized":"2024-09-17T23:00:16.366919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'462","last_scrub_stamp":"2024-09-17T23:00:16.366813+0000","last_deep_scrub":"178'462","last_deep_scrub_stamp":"2024-09-17T23:00:16.366813+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.366813+0000","objects_scrubbed":13,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:16.777551+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00062201400000000005,"stat_sum":{"num_bytes":19,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11078,"num_read_kb":23731,"num_write":419,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'797","reported_seq":5736,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.533530+0000","last_change":"2024-09-17T23:00:15.533596+0000","last_active":"2024-09-17T23:00:15.533530+0000","last_peered":"2024-09-17T23:00:15.533530+0000","last_clean":"2024-09-17T23:00:15.533530+0000","last_became_active":"2024-09-17T22:57:19.848506+0000","last_became_peered":"2024-09-17T22:57:19.848506+0000","last_unstale":"2024-09-17T23:00:15.533530+0000","last_undegraded":"2024-09-17T23:00:15.533530+0000","last_fullsized":"2024-09-17T23:00:15.533530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'797","last_scrub_stamp":"2024-09-17T23:00:15.533428+0000","last_deep_scrub":"178'797","last_deep_scrub_stamp":"2024-09-17T23:00:15.533428+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.533428+0000","objects_scrubbed":15,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:56:37.571611+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001226993,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6852,"num_read_kb":19733,"num_write":871,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'527","reported_seq":5999,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.560406+0000","last_change":"2024-09-17T23:00:17.560478+0000","last_active":"2024-09-17T23:00:17.560406+0000","last_peered":"2024-09-17T23:00:17.560406+0000","last_clean":"2024-09-17T23:00:17.560406+0000","last_became_active":"2024-09-17T22:59:19.034604+0000","last_became_peered":"2024-09-17T22:59:19.034604+0000","last_unstale":"2024-09-17T23:00:17.560406+0000","last_undegraded":"2024-09-17T23:00:17.560406+0000","last_fullsized":"2024-09-17T23:00:17.560406+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'527","last_scrub_stamp":"2024-09-17T23:00:17.560299+0000","last_deep_scrub":"178'527","last_deep_scrub_stamp":"2024-09-17T23:00:17.560299+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.560299+0000","objects_scrubbed":10,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:55:35.033245+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0014018030000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4775,"num_read_kb":27559,"num_write":479,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8489337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'713","reported_seq":4811,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.029152+0000","last_change":"2024-09-17T23:00:14.029191+0000","last_active":"2024-09-17T23:00:14.029152+0000","last_peered":"2024-09-17T23:00:14.029152+0000","last_clean":"2024-09-17T23:00:14.029152+0000","last_became_active":"2024-09-17T22:58:55.861675+0000","last_became_peered":"2024-09-17T22:58:55.861675+0000","last_unstale":"2024-09-17T23:00:14.029152+0000","last_undegraded":"2024-09-17T23:00:14.029152+0000","last_fullsized":"2024-09-17T23:00:14.029152+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'713","last_scrub_stamp":"2024-09-17T23:00:14.029081+0000","last_deep_scrub":"178'713","last_deep_scrub_stamp":"2024-09-17T23:00:14.029081+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.029081+0000","objects_scrubbed":10,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:39.755238+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.002294089,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5624,"num_read_kb":21155,"num_write":832,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5604223,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'1349","reported_seq":9817,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:18.985033+0000","last_change":"2024-09-17T23:00:18.985079+0000","last_active":"2024-09-17T23:00:18.985033+0000","last_peered":"2024-09-17T23:00:18.985033+0000","last_clean":"2024-09-17T23:00:18.985033+0000","last_became_active":"2024-09-17T22:58:55.862661+0000","last_became_peered":"2024-09-17T22:58:55.862661+0000","last_unstale":"2024-09-17T23:00:18.985033+0000","last_undegraded":"2024-09-17T23:00:18.985033+0000","last_fullsized":"2024-09-17T23:00:18.985033+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'1349","last_scrub_stamp":"2024-09-17T23:00:18.984947+0000","last_deep_scrub":"178'1349","last_deep_scrub_stamp":"2024-09-17T23:00:18.984947+0000","last_clean_scrub_stamp":"2024-09-17T23:00:18.984947+0000","objects_scrubbed":16,"log_size":1349,"log_dups_size":0,"ondisk_log_size":1349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:47:26.761012+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.002098108,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13665,"num_read_kb":44276,"num_write":1909,"num_write_kb":13668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":10868111,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'502","reported_seq":5053,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:21.063706+0000","last_change":"2024-09-17T23:00:21.063810+0000","last_active":"2024-09-17T23:00:21.063706+0000","last_peered":"2024-09-17T23:00:21.063706+0000","last_clean":"2024-09-17T23:00:21.063706+0000","last_became_active":"2024-09-17T23:00:02.093523+0000","last_became_peered":"2024-09-17T23:00:02.093523+0000","last_unstale":"2024-09-17T23:00:21.063706+0000","last_undegraded":"2024-09-17T23:00:21.063706+0000","last_fullsized":"2024-09-17T23:00:21.063706+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'502","last_scrub_stamp":"2024-09-17T23:00:21.063603+0000","last_deep_scrub":"178'502","last_deep_scrub_stamp":"2024-09-17T23:00:21.063603+0000","last_clean_scrub_stamp":"2024-09-17T23:00:21.063603+0000","objects_scrubbed":17,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:32.114947+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.002331428,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4744,"num_read_kb":28498,"num_write":440,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'529","reported_seq":3078,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.391460+0000","last_change":"2024-09-17T23:00:15.391547+0000","last_active":"2024-09-17T23:00:15.391460+0000","last_peered":"2024-09-17T23:00:15.391460+0000","last_clean":"2024-09-17T23:00:15.391460+0000","last_became_active":"2024-09-17T22:57:19.852636+0000","last_became_peered":"2024-09-17T22:57:19.852636+0000","last_unstale":"2024-09-17T23:00:15.391460+0000","last_undegraded":"2024-09-17T23:00:15.391460+0000","last_fullsized":"2024-09-17T23:00:15.391460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'529","last_scrub_stamp":"2024-09-17T23:00:15.391358+0000","last_deep_scrub":"178'529","last_deep_scrub_stamp":"2024-09-17T23:00:15.391358+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.391358+0000","objects_scrubbed":20,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:28.249106+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00058748600000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3563,"num_read_kb":11481,"num_write":487,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":355,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.618621+0000","last_change":"2024-09-17T23:00:17.618647+0000","last_active":"2024-09-17T23:00:17.618621+0000","last_peered":"2024-09-17T23:00:17.618621+0000","last_clean":"2024-09-17T23:00:17.618621+0000","last_became_active":"2024-09-17T22:59:21.869594+0000","last_became_peered":"2024-09-17T22:59:21.869594+0000","last_unstale":"2024-09-17T23:00:17.618621+0000","last_undegraded":"2024-09-17T23:00:17.618621+0000","last_fullsized":"2024-09-17T23:00:17.618621+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:17.618525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:17.618525+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.618525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:25.325686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045280800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'710","reported_seq":10521,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:13.021274+0000","last_change":"2024-09-17T23:00:13.021326+0000","last_active":"2024-09-17T23:00:13.021274+0000","last_peered":"2024-09-17T23:00:13.021274+0000","last_clean":"2024-09-17T23:00:13.021274+0000","last_became_active":"2024-09-17T22:57:19.851250+0000","last_became_peered":"2024-09-17T22:57:19.851250+0000","last_unstale":"2024-09-17T23:00:13.021274+0000","last_undegraded":"2024-09-17T23:00:13.021274+0000","last_fullsized":"2024-09-17T23:00:13.021274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'710","last_scrub_stamp":"2024-09-17T23:00:13.021189+0000","last_deep_scrub":"178'710","last_deep_scrub_stamp":"2024-09-17T23:00:13.021189+0000","last_clean_scrub_stamp":"2024-09-17T23:00:13.021189+0000","objects_scrubbed":15,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:48.476667+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001273291,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12436,"num_read_kb":26526,"num_write":569,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":342,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:13.469433+0000","last_change":"2024-09-17T23:00:13.469455+0000","last_active":"2024-09-17T23:00:13.469433+0000","last_peered":"2024-09-17T23:00:13.469433+0000","last_clean":"2024-09-17T23:00:13.469433+0000","last_became_active":"2024-09-17T22:57:09.890284+0000","last_became_peered":"2024-09-17T22:57:09.890284+0000","last_unstale":"2024-09-17T23:00:13.469433+0000","last_undegraded":"2024-09-17T23:00:13.469433+0000","last_fullsized":"2024-09-17T23:00:13.469433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:13.469346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:13.469346+0000","last_clean_scrub_stamp":"2024-09-17T23:00:13.469346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:57:40.989221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043822499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'551","reported_seq":9954,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:13.526489+0000","last_change":"2024-09-17T23:00:13.526559+0000","last_active":"2024-09-17T23:00:13.526489+0000","last_peered":"2024-09-17T23:00:13.526489+0000","last_clean":"2024-09-17T23:00:13.526489+0000","last_became_active":"2024-09-17T22:57:19.844834+0000","last_became_peered":"2024-09-17T22:57:19.844834+0000","last_unstale":"2024-09-17T23:00:13.526489+0000","last_undegraded":"2024-09-17T23:00:13.526489+0000","last_fullsized":"2024-09-17T23:00:13.526489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'551","last_scrub_stamp":"2024-09-17T23:00:13.526376+0000","last_deep_scrub":"178'551","last_deep_scrub_stamp":"2024-09-17T23:00:13.526376+0000","last_clean_scrub_stamp":"2024-09-17T23:00:13.526376+0000","objects_scrubbed":18,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:10.134155+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.002292189,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9866,"num_read_kb":29906,"num_write":527,"num_write_kb":5481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":342,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.249185+0000","last_change":"2024-09-17T23:00:15.249211+0000","last_active":"2024-09-17T23:00:15.249185+0000","last_peered":"2024-09-17T23:00:15.249185+0000","last_clean":"2024-09-17T23:00:15.249185+0000","last_became_active":"2024-09-17T22:57:09.996664+0000","last_became_peered":"2024-09-17T22:57:09.996664+0000","last_unstale":"2024-09-17T23:00:15.249185+0000","last_undegraded":"2024-09-17T23:00:15.249185+0000","last_fullsized":"2024-09-17T23:00:15.249185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:15.249085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:15.249085+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.249085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:31:04.997364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047543500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"178'616","reported_seq":10124,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.528416+0000","last_change":"2024-09-17T23:00:16.528473+0000","last_active":"2024-09-17T23:00:16.528416+0000","last_peered":"2024-09-17T23:00:16.528416+0000","last_clean":"2024-09-17T23:00:16.528416+0000","last_became_active":"2024-09-17T22:59:10.952505+0000","last_became_peered":"2024-09-17T22:59:10.952505+0000","last_unstale":"2024-09-17T23:00:16.528416+0000","last_undegraded":"2024-09-17T23:00:16.528416+0000","last_fullsized":"2024-09-17T23:00:16.528416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'616","last_scrub_stamp":"2024-09-17T23:00:16.528316+0000","last_deep_scrub":"178'616","last_deep_scrub_stamp":"2024-09-17T23:00:16.528316+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.528316+0000","objects_scrubbed":20,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:29.237898+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0012959169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":10838,"num_read_kb":27436,"num_write":545,"num_write_kb":10554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":4212294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":342,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.502119+0000","last_change":"2024-09-17T23:00:14.502140+0000","last_active":"2024-09-17T23:00:14.502119+0000","last_peered":"2024-09-17T23:00:14.502119+0000","last_clean":"2024-09-17T23:00:14.502119+0000","last_became_active":"2024-09-17T22:57:09.996575+0000","last_became_peered":"2024-09-17T22:57:09.996575+0000","last_unstale":"2024-09-17T23:00:14.502119+0000","last_undegraded":"2024-09-17T23:00:14.502119+0000","last_fullsized":"2024-09-17T23:00:14.502119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:14.502054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:14.502054+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.502054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:39:05.778639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044797499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"178'788","reported_seq":4277,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:20.002800+0000","last_change":"2024-09-17T23:00:20.002861+0000","last_active":"2024-09-17T23:00:20.002800+0000","last_peered":"2024-09-17T23:00:20.002800+0000","last_clean":"2024-09-17T23:00:20.002800+0000","last_became_active":"2024-09-17T22:58:56.863971+0000","last_became_peered":"2024-09-17T22:58:56.863971+0000","last_unstale":"2024-09-17T23:00:20.002800+0000","last_undegraded":"2024-09-17T23:00:20.002800+0000","last_fullsized":"2024-09-17T23:00:20.002800+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'788","last_scrub_stamp":"2024-09-17T23:00:20.002691+0000","last_deep_scrub":"178'788","last_deep_scrub_stamp":"2024-09-17T23:00:20.002691+0000","last_clean_scrub_stamp":"2024-09-17T23:00:20.002691+0000","objects_scrubbed":8,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:46.331956+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0011476489999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6022,"num_read_kb":19563,"num_write":935,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":7678335,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'767","reported_seq":5533,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.394200+0000","last_change":"2024-09-17T23:00:14.394262+0000","last_active":"2024-09-17T23:00:14.394200+0000","last_peered":"2024-09-17T23:00:14.394200+0000","last_clean":"2024-09-17T23:00:14.394200+0000","last_became_active":"2024-09-17T22:57:19.851489+0000","last_became_peered":"2024-09-17T22:57:19.851489+0000","last_unstale":"2024-09-17T23:00:14.394200+0000","last_undegraded":"2024-09-17T23:00:14.394200+0000","last_fullsized":"2024-09-17T23:00:14.394200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'767","last_scrub_stamp":"2024-09-17T23:00:14.394101+0000","last_deep_scrub":"178'767","last_deep_scrub_stamp":"2024-09-17T23:00:14.394101+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.394101+0000","objects_scrubbed":15,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:49:39.840624+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001048319,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5446,"num_read_kb":24042,"num_write":734,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"178'3","reported_seq":168,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.018843+0000","last_change":"2024-09-17T23:00:15.018872+0000","last_active":"2024-09-17T23:00:15.018843+0000","last_peered":"2024-09-17T23:00:15.018843+0000","last_clean":"2024-09-17T23:00:15.018843+0000","last_became_active":"2024-09-17T22:58:52.181150+0000","last_became_peered":"2024-09-17T22:58:52.181150+0000","last_unstale":"2024-09-17T23:00:15.018843+0000","last_undegraded":"2024-09-17T23:00:15.018843+0000","last_fullsized":"2024-09-17T23:00:15.018843+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'3","last_scrub_stamp":"2024-09-17T23:00:15.018724+0000","last_deep_scrub":"178'3","last_deep_scrub_stamp":"2024-09-17T23:00:15.018724+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.018724+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:03.899556+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'421","reported_seq":19339,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.515684+0000","last_change":"2024-09-17T23:00:17.515939+0000","last_active":"2024-09-17T23:00:17.515684+0000","last_peered":"2024-09-17T23:00:17.515684+0000","last_clean":"2024-09-17T23:00:17.515684+0000","last_became_active":"2024-09-17T22:59:13.979211+0000","last_became_peered":"2024-09-17T22:59:13.979211+0000","last_unstale":"2024-09-17T23:00:17.515684+0000","last_undegraded":"2024-09-17T23:00:17.515684+0000","last_fullsized":"2024-09-17T23:00:17.515684+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'421","last_scrub_stamp":"2024-09-17T23:00:17.515593+0000","last_deep_scrub":"178'421","last_deep_scrub_stamp":"2024-09-17T23:00:17.515593+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.515593+0000","objects_scrubbed":11,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:54.597080+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0016376780000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":19636,"num_read_kb":25949,"num_write":354,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":249018,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:12.946425+0000","last_change":"2024-09-17T23:00:12.946462+0000","last_active":"2024-09-17T23:00:12.946425+0000","last_peered":"2024-09-17T23:00:12.946425+0000","last_clean":"2024-09-17T23:00:12.946425+0000","last_became_active":"2024-09-17T22:59:10.964430+0000","last_became_peered":"2024-09-17T22:59:10.964430+0000","last_unstale":"2024-09-17T23:00:12.946425+0000","last_undegraded":"2024-09-17T23:00:12.946425+0000","last_fullsized":"2024-09-17T23:00:12.946425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:12.946338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:12.946338+0000","last_clean_scrub_stamp":"2024-09-17T23:00:12.946338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:18.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'1278","reported_seq":13375,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.020063+0000","last_change":"2024-09-17T23:00:17.020115+0000","last_active":"2024-09-17T23:00:17.020063+0000","last_peered":"2024-09-17T23:00:17.020063+0000","last_clean":"2024-09-17T23:00:17.020063+0000","last_became_active":"2024-09-17T22:59:21.867657+0000","last_became_peered":"2024-09-17T22:59:21.867657+0000","last_unstale":"2024-09-17T23:00:17.020063+0000","last_undegraded":"2024-09-17T23:00:17.020063+0000","last_fullsized":"2024-09-17T23:00:17.020063+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'1278","last_scrub_stamp":"2024-09-17T23:00:17.019951+0000","last_deep_scrub":"178'1278","last_deep_scrub_stamp":"2024-09-17T23:00:17.019951+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.019951+0000","objects_scrubbed":15,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:49:42.615434+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0010198220000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16206,"num_read_kb":43141,"num_write":1909,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1289362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:18.131718+0000","last_change":"2024-09-17T23:00:18.131744+0000","last_active":"2024-09-17T23:00:18.131718+0000","last_peered":"2024-09-17T23:00:18.131718+0000","last_clean":"2024-09-17T23:00:18.131718+0000","last_became_active":"2024-09-17T23:00:02.093740+0000","last_became_peered":"2024-09-17T23:00:02.093740+0000","last_unstale":"2024-09-17T23:00:18.131718+0000","last_undegraded":"2024-09-17T23:00:18.131718+0000","last_fullsized":"2024-09-17T23:00:18.131718+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:00:18.131612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:00:18.131612+0000","last_clean_scrub_stamp":"2024-09-17T23:00:18.131612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:30.835890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'872","reported_seq":6347,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.533639+0000","last_change":"2024-09-17T23:00:15.533693+0000","last_active":"2024-09-17T23:00:15.533639+0000","last_peered":"2024-09-17T23:00:15.533639+0000","last_clean":"2024-09-17T23:00:15.533639+0000","last_became_active":"2024-09-17T22:58:55.860905+0000","last_became_peered":"2024-09-17T22:58:55.860905+0000","last_unstale":"2024-09-17T23:00:15.533639+0000","last_undegraded":"2024-09-17T23:00:15.533639+0000","last_fullsized":"2024-09-17T23:00:15.533639+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'872","last_scrub_stamp":"2024-09-17T23:00:15.533574+0000","last_deep_scrub":"178'872","last_deep_scrub_stamp":"2024-09-17T23:00:15.533574+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.533574+0000","objects_scrubbed":12,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:12.268704+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0019368829999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7033,"num_read_kb":16026,"num_write":1087,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":3469207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"178'2","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:21.023794+0000","last_change":"2024-09-17T23:00:21.023831+0000","last_active":"2024-09-17T23:00:21.023794+0000","last_peered":"2024-09-17T23:00:21.023794+0000","last_clean":"2024-09-17T23:00:21.023794+0000","last_became_active":"2024-09-17T22:59:11.956110+0000","last_became_peered":"2024-09-17T22:59:11.956110+0000","last_unstale":"2024-09-17T23:00:21.023794+0000","last_undegraded":"2024-09-17T23:00:21.023794+0000","last_fullsized":"2024-09-17T23:00:21.023794+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'2","last_scrub_stamp":"2024-09-17T23:00:21.023682+0000","last_deep_scrub":"178'2","last_deep_scrub_stamp":"2024-09-17T23:00:21.023682+0000","last_clean_scrub_stamp":"2024-09-17T23:00:21.023682+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:52.510927+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"178'119","reported_seq":349,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:20.105109+0000","last_change":"2024-09-17T23:00:20.105163+0000","last_active":"2024-09-17T23:00:20.105109+0000","last_peered":"2024-09-17T23:00:20.105109+0000","last_clean":"2024-09-17T23:00:20.105109+0000","last_became_active":"2024-09-17T23:00:02.094535+0000","last_became_peered":"2024-09-17T23:00:02.094535+0000","last_unstale":"2024-09-17T23:00:20.105109+0000","last_undegraded":"2024-09-17T23:00:20.105109+0000","last_fullsized":"2024-09-17T23:00:20.105109+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'119","last_scrub_stamp":"2024-09-17T23:00:20.104994+0000","last_deep_scrub":"178'119","last_deep_scrub_stamp":"2024-09-17T23:00:20.104994+0000","last_clean_scrub_stamp":"2024-09-17T23:00:20.104994+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:00.447286+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'829","reported_seq":4026,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:18.611072+0000","last_change":"2024-09-17T23:00:18.611120+0000","last_active":"2024-09-17T23:00:18.611072+0000","last_peered":"2024-09-17T23:00:18.611072+0000","last_clean":"2024-09-17T23:00:18.611072+0000","last_became_active":"2024-09-17T22:59:21.869955+0000","last_became_peered":"2024-09-17T22:59:21.869955+0000","last_unstale":"2024-09-17T23:00:18.611072+0000","last_undegraded":"2024-09-17T23:00:18.611072+0000","last_fullsized":"2024-09-17T23:00:18.611072+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'829","last_scrub_stamp":"2024-09-17T23:00:18.610977+0000","last_deep_scrub":"178'829","last_deep_scrub_stamp":"2024-09-17T23:00:18.610977+0000","last_clean_scrub_stamp":"2024-09-17T23:00:18.610977+0000","objects_scrubbed":12,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:09.760864+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001050664,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5381,"num_read_kb":13695,"num_write":1120,"num_write_kb":10151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":4301443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'572","reported_seq":4362,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:15.662194+0000","last_change":"2024-09-17T23:00:15.662236+0000","last_active":"2024-09-17T23:00:15.662194+0000","last_peered":"2024-09-17T23:00:15.662194+0000","last_clean":"2024-09-17T23:00:15.662194+0000","last_became_active":"2024-09-17T22:57:34.006700+0000","last_became_peered":"2024-09-17T22:57:34.006700+0000","last_unstale":"2024-09-17T23:00:15.662194+0000","last_undegraded":"2024-09-17T23:00:15.662194+0000","last_fullsized":"2024-09-17T23:00:15.662194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'572","last_scrub_stamp":"2024-09-17T23:00:15.662107+0000","last_deep_scrub":"178'572","last_deep_scrub_stamp":"2024-09-17T23:00:15.662107+0000","last_clean_scrub_stamp":"2024-09-17T23:00:15.662107+0000","objects_scrubbed":10,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:41.867614+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00070571200000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5472,"num_read_kb":14550,"num_write":529,"num_write_kb":5130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":170949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"178'580","reported_seq":3713,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:14.658988+0000","last_change":"2024-09-17T23:00:14.659045+0000","last_active":"2024-09-17T23:00:14.658988+0000","last_peered":"2024-09-17T23:00:14.658988+0000","last_clean":"2024-09-17T23:00:14.658988+0000","last_became_active":"2024-09-17T22:57:19.855104+0000","last_became_peered":"2024-09-17T22:57:19.855104+0000","last_unstale":"2024-09-17T23:00:14.658988+0000","last_undegraded":"2024-09-17T23:00:14.658988+0000","last_fullsized":"2024-09-17T23:00:14.658988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'580","last_scrub_stamp":"2024-09-17T23:00:14.658880+0000","last_deep_scrub":"178'580","last_deep_scrub_stamp":"2024-09-17T23:00:14.658880+0000","last_clean_scrub_stamp":"2024-09-17T23:00:14.658880+0000","objects_scrubbed":16,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:50:46.735418+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00051363500000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3627,"num_read_kb":19060,"num_write":642,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'575","reported_seq":6550,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.573979+0000","last_change":"2024-09-17T23:00:16.574064+0000","last_active":"2024-09-17T23:00:16.573979+0000","last_peered":"2024-09-17T23:00:16.573979+0000","last_clean":"2024-09-17T23:00:16.573979+0000","last_became_active":"2024-09-17T22:59:19.031794+0000","last_became_peered":"2024-09-17T22:59:19.031794+0000","last_unstale":"2024-09-17T23:00:16.573979+0000","last_undegraded":"2024-09-17T23:00:16.573979+0000","last_fullsized":"2024-09-17T23:00:16.573979+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'575","last_scrub_stamp":"2024-09-17T23:00:16.573864+0000","last_deep_scrub":"178'575","last_deep_scrub_stamp":"2024-09-17T23:00:16.573864+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.573864+0000","objects_scrubbed":15,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:38:14.527584+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001368881,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6523,"num_read_kb":16592,"num_write":536,"num_write_kb":10984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4929678,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'441","reported_seq":5283,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.987420+0000","last_change":"2024-09-17T23:00:17.987463+0000","last_active":"2024-09-17T23:00:17.987420+0000","last_peered":"2024-09-17T23:00:17.987420+0000","last_clean":"2024-09-17T23:00:17.987420+0000","last_became_active":"2024-09-17T22:58:54.865392+0000","last_became_peered":"2024-09-17T22:58:54.865392+0000","last_unstale":"2024-09-17T23:00:17.987420+0000","last_undegraded":"2024-09-17T23:00:17.987420+0000","last_fullsized":"2024-09-17T23:00:17.987420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'441","last_scrub_stamp":"2024-09-17T23:00:17.987335+0000","last_deep_scrub":"178'441","last_deep_scrub_stamp":"2024-09-17T23:00:17.987335+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.987335+0000","objects_scrubbed":13,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:49.883440+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00066149199999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5440,"num_read_kb":18887,"num_write":481,"num_write_kb":10847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":9594465,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'459","reported_seq":5137,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:22.037183+0000","last_change":"2024-09-17T23:00:22.037220+0000","last_active":"2024-09-17T23:00:22.037183+0000","last_peered":"2024-09-17T23:00:22.037183+0000","last_clean":"2024-09-17T23:00:22.037183+0000","last_became_active":"2024-09-17T23:00:02.095583+0000","last_became_peered":"2024-09-17T23:00:02.095583+0000","last_unstale":"2024-09-17T23:00:22.037183+0000","last_undegraded":"2024-09-17T23:00:22.037183+0000","last_fullsized":"2024-09-17T23:00:22.037183+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'459","last_scrub_stamp":"2024-09-17T23:00:22.037087+0000","last_deep_scrub":"178'459","last_deep_scrub_stamp":"2024-09-17T23:00:22.037087+0000","last_clean_scrub_stamp":"2024-09-17T23:00:22.037087+0000","objects_scrubbed":10,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:21.368995+0000","scrub_duration":8,"objects_trimmed":1,"snaptrim_duration":0.0041489919999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6347,"num_read_kb":23723,"num_write":441,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":3096453,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'369","reported_seq":2222,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:17.188506+0000","last_change":"2024-09-17T23:00:17.188560+0000","last_active":"2024-09-17T23:00:17.188506+0000","last_peered":"2024-09-17T23:00:17.188506+0000","last_clean":"2024-09-17T23:00:17.188506+0000","last_became_active":"2024-09-17T23:00:02.092220+0000","last_became_peered":"2024-09-17T23:00:02.092220+0000","last_unstale":"2024-09-17T23:00:17.188506+0000","last_undegraded":"2024-09-17T23:00:17.188506+0000","last_fullsized":"2024-09-17T23:00:17.188506+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'369","last_scrub_stamp":"2024-09-17T23:00:17.188421+0000","last_deep_scrub":"178'369","last_deep_scrub_stamp":"2024-09-17T23:00:17.188421+0000","last_clean_scrub_stamp":"2024-09-17T23:00:17.188421+0000","objects_scrubbed":17,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:15:50.983638+0000","scrub_duration":11,"objects_trimmed":2,"snaptrim_duration":0.000759419,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1608,"num_read_kb":7804,"num_write":333,"num_write_kb":8446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":3668483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'1114","reported_seq":9339,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:19.125588+0000","last_change":"2024-09-17T23:00:19.125677+0000","last_active":"2024-09-17T23:00:19.125588+0000","last_peered":"2024-09-17T23:00:19.125588+0000","last_clean":"2024-09-17T23:00:19.125588+0000","last_became_active":"2024-09-17T23:00:02.091943+0000","last_became_peered":"2024-09-17T23:00:02.091943+0000","last_unstale":"2024-09-17T23:00:19.125588+0000","last_undegraded":"2024-09-17T23:00:19.125588+0000","last_fullsized":"2024-09-17T23:00:19.125588+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'1114","last_scrub_stamp":"2024-09-17T23:00:19.125475+0000","last_deep_scrub":"178'1114","last_deep_scrub_stamp":"2024-09-17T23:00:19.125475+0000","last_clean_scrub_stamp":"2024-09-17T23:00:19.125475+0000","objects_scrubbed":16,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:44:07.739358+0000","scrub_duration":13,"objects_trimmed":2,"snaptrim_duration":0.0049515920000000003,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12334,"num_read_kb":34887,"num_write":1395,"num_write_kb":49279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"178'715","reported_seq":5856,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.671913+0000","last_change":"2024-09-17T23:00:16.671953+0000","last_active":"2024-09-17T23:00:16.671913+0000","last_peered":"2024-09-17T23:00:16.671913+0000","last_clean":"2024-09-17T23:00:16.671913+0000","last_became_active":"2024-09-17T22:59:19.034656+0000","last_became_peered":"2024-09-17T22:59:19.034656+0000","last_unstale":"2024-09-17T23:00:16.671913+0000","last_undegraded":"2024-09-17T23:00:16.671913+0000","last_fullsized":"2024-09-17T23:00:16.671913+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'715","last_scrub_stamp":"2024-09-17T23:00:16.671814+0000","last_deep_scrub":"178'715","last_deep_scrub_stamp":"2024-09-17T23:00:16.671814+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.671814+0000","objects_scrubbed":11,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:48.253804+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00043466700000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5722,"num_read_kb":33563,"num_write":880,"num_write_kb":14921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":14040474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"178'554","reported_seq":4765,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:22.019917+0000","last_change":"2024-09-17T23:00:22.019978+0000","last_active":"2024-09-17T23:00:22.019917+0000","last_peered":"2024-09-17T23:00:22.019917+0000","last_clean":"2024-09-17T23:00:22.019917+0000","last_became_active":"2024-09-17T22:59:13.979290+0000","last_became_peered":"2024-09-17T22:59:13.979290+0000","last_unstale":"2024-09-17T23:00:22.019917+0000","last_undegraded":"2024-09-17T23:00:22.019917+0000","last_fullsized":"2024-09-17T23:00:22.019917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'554","last_scrub_stamp":"2024-09-17T23:00:22.019832+0000","last_deep_scrub":"178'554","last_deep_scrub_stamp":"2024-09-17T23:00:22.019832+0000","last_clean_scrub_stamp":"2024-09-17T23:00:22.019832+0000","objects_scrubbed":22,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:52.968661+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0010121710000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4772,"num_read_kb":17301,"num_write":495,"num_write_kb":5651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":3774763,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"178'820","reported_seq":13597,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:12.067342+0000","last_change":"2024-09-17T23:00:12.067380+0000","last_active":"2024-09-17T23:00:12.067342+0000","last_peered":"2024-09-17T23:00:12.067342+0000","last_clean":"2024-09-17T23:00:12.067342+0000","last_became_active":"2024-09-17T22:57:19.850719+0000","last_became_peered":"2024-09-17T22:57:19.850719+0000","last_unstale":"2024-09-17T23:00:12.067342+0000","last_undegraded":"2024-09-17T23:00:12.067342+0000","last_fullsized":"2024-09-17T23:00:12.067342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'820","last_scrub_stamp":"2024-09-17T23:00:12.067262+0000","last_deep_scrub":"178'820","last_deep_scrub_stamp":"2024-09-17T23:00:12.067262+0000","last_clean_scrub_stamp":"2024-09-17T23:00:12.067262+0000","objects_scrubbed":15,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:10.115263+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00079959200000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15329,"num_read_kb":33221,"num_write":992,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"178'695","reported_seq":9279,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-17T23:00:16.220519+0000","last_change":"2024-09-17T23:00:16.220613+0000","last_active":"2024-09-17T23:00:16.220519+0000","last_peered":"2024-09-17T23:00:16.220519+0000","last_clean":"2024-09-17T23:00:16.220519+0000","last_became_active":"2024-09-17T22:57:19.852295+0000","last_became_peered":"2024-09-17T22:57:19.852295+0000","last_unstale":"2024-09-17T23:00:16.220519+0000","last_undegraded":"2024-09-17T23:00:16.220519+0000","last_fullsized":"2024-09-17T23:00:16.220519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'695","last_scrub_stamp":"2024-09-17T23:00:16.220418+0000","last_deep_scrub":"178'695","last_deep_scrub_stamp":"2024-09-17T23:00:16.220418+0000","last_clean_scrub_stamp":"2024-09-17T23:00:16.220418+0000","objects_scrubbed":21,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:05:04.107879+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00053445299999999995,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10608,"num_read_kb":26844,"num_write":727,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":444,"num_read":262266,"num_read_kb":796557,"num_write":23452,"num_write_kb":366334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3850,"num_bytes_recovered":119485722,"num_keys_recovered":546,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16139,"internal_metadata":0},"log_size":21469,"ondisk_log_size":21469,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29112,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342728,"statfs":{"total":96636764160,"available":96606953472,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":71,"seq":304942678047,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30908,"kb_used_data":2092,"kb_used_omap":17,"kb_used_meta":28782,"kb_avail":94340932,"statfs":{"total":96636764160,"available":96605114368,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18234,"internal_metadata":29472966},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2142208,"data_stored":1042306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59640,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312200,"statfs":{"total":96636764160,"available":96575692800,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":123,"seq":528280977427,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34680,"kb_used_data":2088,"kb_used_omap":14,"kb_used_meta":32561,"kb_avail":94337160,"statfs":{"total":96636764160,"available":96601251840,"internally_reserved":0,"allocated":2138112,"data_stored":1042287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14934,"internal_metadata":33342890},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2146304,"data_stored":1042325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65788,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306052,"statfs":{"total":96636764160,"available":96569397248,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":2732032,"data_stored":1632655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10558,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:00:35.873 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T23:00:36.167 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T23:00:36.168 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T23:00:36.168 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T23:00:36.168 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.212 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T23:00:36.212 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T23:00:36.212 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.251 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T23:00:36.252 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T23:00:36.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.325 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T23:00:36.326 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T23:00:36.326 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.396 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T23:00:36.396 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T23:00:36.396 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.448 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T23:00:36.448 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T23:00:36.448 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.485 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T23:00:36.486 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T23:00:36.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.559 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T23:00:36.559 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T23:00:36.559 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.633 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T23:00:36.633 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T23:00:36.633 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T23:00:36.633 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.657 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T23:00:36.657 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T23:00:36.657 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.704 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T23:00:36.704 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T23:00:36.704 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T23:00:36.704 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.716 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T23:00:36.716 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T23:00:36.716 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.768 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T23:00:36.768 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T23:00:36.768 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:00:36.780 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T23:00:36.780 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T23:00:36.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T23:00:36.840 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T23:00:36.840 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T23:00:37.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T23:00:37.057 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T23:00:37.182 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T23:00:37.183 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T23:00:37.228 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T23:00:37.229 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T23:00:37.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T23:00:37.281 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T23:00:37.428 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T23:00:37.429 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T23:00:37.500 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T23:00:37.501 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T23:00:37.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T23:00:37.567 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T23:00:37.627 INFO:tasks.ceph:Archiving mon data... 2024-09-17T23:00:37.628 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/data/mon.a.tgz 2024-09-17T23:00:37.628 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-09-17T23:00:37.644 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.odc4LTd35h 2024-09-17T23:00:37.644 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.odc4LTd35h 2024-09-17T23:00:37.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.odc4LTd35h 2024-09-17T23:00:37.890 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.odc4LTd35h is 413KB 2024-09-17T23:00:37.959 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.odc4LTd35h 2024-09-17T23:00:37.975 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/data/mon.c.tgz 2024-09-17T23:00:37.975 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-09-17T23:00:38.029 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.lgkzuV2B7u 2024-09-17T23:00:38.029 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lgkzuV2B7u 2024-09-17T23:00:38.207 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.lgkzuV2B7u 2024-09-17T23:00:38.290 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.lgkzuV2B7u is 557KB 2024-09-17T23:00:38.389 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.lgkzuV2B7u 2024-09-17T23:00:38.404 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/data/mon.b.tgz 2024-09-17T23:00:38.405 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-17T23:00:38.424 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.q0IPPlItJJ 2024-09-17T23:00:38.424 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.q0IPPlItJJ 2024-09-17T23:00:38.622 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.q0IPPlItJJ 2024-09-17T23:00:38.705 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.q0IPPlItJJ is 557KB 2024-09-17T23:00:38.776 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.q0IPPlItJJ 2024-09-17T23:00:38.791 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T23:00:38.792 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T23:00:38.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T23:00:38.871 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T23:00:38.911 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T23:00:38.911 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T23:00:38.913 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi159/crash 2024-09-17T23:00:38.913 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T23:00:38.938 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi171/crash 2024-09-17T23:00:38.939 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T23:00:38.974 INFO:tasks.ceph:Compressing logs... 2024-09-17T23:00:38.975 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T23:00:38.982 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T23:00:39.061 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32896.log 2024-09-17T23:00:39.061 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T23:00:39.061 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32809.log 2024-09-17T23:00:39.061 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32896.log.gz 2024-09-17T23:00:39.061 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32809.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42767.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32809.log.gz 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/objectstore_tool.42767.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40127.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32976.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/objectstore_tool.40127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33096.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32976.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32976.log.gz 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32856.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33096.log.gz 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32936.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33016.log 2024-09-17T23:00:39.062 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32856.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33056.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.32936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32936.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33016.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33016.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.33056.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33056.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41867.log 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mgr.x.log: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.42767.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.40127.log.gz 2024-09-17T23:00:39.063 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T23:00:39.064 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/objectstore_tool.41867.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41067.log 2024-09-17T23:00:39.068 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/objectstore_tool.41067.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.41867.log.gz 2024-09-17T23:00:39.068 INFO:teuthology.orchestra.run.smithi171.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.41067.log.gz 2024-09-17T23:00:39.116 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T23:00:39.116 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-09-17T23:00:39.116 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38853.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53077.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58848.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38853.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5/var/log/ceph/ceph-client.admin.53077.log: --verbose -- /var/log/ceph/ceph-client.admin.46660.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53077.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39979.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58848.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46660.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62946.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39979.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38857.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-client.admin.41296.log 2024-09-17T23:00:39.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41254.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62946.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59369.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61542.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41254.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59369.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62218.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61542.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59008.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38372.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62218.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62944.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59008.log.gz 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42581.log 2024-09-17T23:00:39.118 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38372.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57917.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62944.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55198.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42581.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33672.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57917.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39490.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55198.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.33672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58946.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33672.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60440.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39490.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61713.log 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58946.log.gz 2024-09-17T23:00:39.119 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60440.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39150.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61713.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52839.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5/var/log/ceph/ceph-client.admin.39150.log: --verbose -- /var/log/ceph/ceph-client.admin.39811.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39150.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59426.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52839.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39811.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39175.log 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59426.log.gz 2024-09-17T23:00:39.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54918.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57735.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41096.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39175.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63355.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57735.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63940.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41096.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54483.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63355.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62302.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63940.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54483.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64444.log 2024-09-17T23:00:39.121 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62302.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41678.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64444.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40443.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41678.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54722.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33579.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40443.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 0.0% 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54722.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.33579.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27429.log 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33579.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-09-17T23:00:39.122 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph.tmp-client.admin.27429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63814.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27429.log.gz 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45532.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz /var/log/ceph/ceph-client.admin.43693.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63814.log.gz 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43693.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43693.log.gz 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45100.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40871.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45100.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45100.log.gz 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42419.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63202.log 2024-09-17T23:00:39.123 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40871.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42419.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39487.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63202.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39491.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57331.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39487.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39491.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63353.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57331.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46148.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64024.log 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63353.log.gz 2024-09-17T23:00:39.124 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46148.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64024.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.64024.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39769.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60659.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39769.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39769.log.gz --verbose 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-client.admin.58019.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62606.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60659.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58019.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62606.log.gz 2024-09-17T23:00:39.125 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64526.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40814.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42200.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64526.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60084.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40814.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53769.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42200.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63354.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60084.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47590.log 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53769.log.gz 2024-09-17T23:00:39.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63354.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.63354.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58565.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39937.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47590.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63772.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58565.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60800.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39937.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63772.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48860.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63772.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62846.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60800.log.gz 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/objectstore_tool.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47365.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59785.log 2024-09-17T23:00:39.127 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62846.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47365.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47365.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61797.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59785.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61797.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40094.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39486.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56616.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.40094.log: /var/log/ceph/ceph-client.admin.39486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log 2024-09-17T23:00:39.128 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39486.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56616.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.62091.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56616.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip 95.3% -- replaced with /var/log/ceph/ceph-client.0.40094.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40041.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62600.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40041.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62091.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43137.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39174.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62600.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43137.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39174.log.gz 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52446.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38635.log 2024-09-17T23:00:39.129 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52446.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52446.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40310.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.48860.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64402.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38635.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54122.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40310.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64402.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64402.log.gz -5 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44390.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61124.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54122.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44390.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44390.log.gz 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63520.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58523.log 2024-09-17T23:00:39.130 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61124.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63520.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.63520.log.gz /var/log/ceph/ceph-client.admin.47134.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60562.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58523.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47134.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47134.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60562.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56970.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.33277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58382.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41398.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56970.log.gz 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58297.log 2024-09-17T23:00:39.131 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58382.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41398.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63646.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58297.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39157.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64066.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63646.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr: 26.2% -- replaced with /var/log/ceph/ceph-client.admin.63646.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38855.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39157.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38859.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64066.log.gz 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44936.log 2024-09-17T23:00:39.132 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38855.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38859.log.gzgzip 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62945.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62386.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44936.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61671.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62945.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62386.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61671.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.61671.log.gz -5 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47412.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53395.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39895.log 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47412.log.gz 2024-09-17T23:00:39.133 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62602.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53395.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39895.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39161.log -- replaced with /var/log/ceph/ceph-client.admin.39895.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39164.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62602.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39161.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56913.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38593.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39164.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55545.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56913.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38593.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62344.log 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55545.log.gz 2024-09-17T23:00:39.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62344.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.62344.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.54463.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62605.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54463.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39391.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62605.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62428.log 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39391.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39391.log.gz -5 2024-09-17T23:00:39.135 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59548.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62428.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62844.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59548.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62610.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41213.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62844.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62610.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62610.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57614.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41213.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57614.log.gz 2024-09-17T23:00:39.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54283.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52722.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57052.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54283.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52722.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57855.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62599.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57052.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53294.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57855.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62599.log.gz 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39853.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54604.log 2024-09-17T23:00:39.137 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53294.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61423.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39853.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54604.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64192.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61423.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58988.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64192.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58443.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62942.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58988.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53728.log 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58443.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62942.log.gz 2024-09-17T23:00:39.138 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61817.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57330.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53728.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38858.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61817.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57330.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52053.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38858.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49796.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52053.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52053.log.gz 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-mon.a.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/objectstore_tool.49796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61923.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57833.log 2024-09-17T23:00:39.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61923.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61881.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57833.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61881.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39531.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39531.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53865.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39531.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43294.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53865.log: /var/log/ceph/ceph-client.admin.54047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62607.log 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gz 2024-09-17T23:00:39.140 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53865.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43294.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47823.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62607.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38517.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/objectstore_tool.47823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44545.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38517.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.49796.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62260.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44545.log: 0.0%/var/log/ceph/ceph-client.admin.43616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55590.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44545.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62260.log.gz 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38542.log 2024-09-17T23:00:39.141 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55590.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55590.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64486.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38542.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64486.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64486.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44074.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61042.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44074.log: /var/log/ceph/ceph-client.admin.61042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44074.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61042.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45911.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54866.log 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-09-17T23:00:39.142 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45911.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62596.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55143.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.47823.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38461.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62596.log: /var/log/ceph/ceph-client.admin.55143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62596.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55143.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64234.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38461.log: /var/log/ceph/ceph-client.admin.64234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64234.log.gz -- replaced with /var/log/ceph/ceph-client.admin.38461.log.gz 2024-09-17T23:00:39.143 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51831.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61965.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42074.log: /var/log/ceph/ceph-client.admin.51831.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51831.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61965.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55414.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64360.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.55414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61084.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55414.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64360.log: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.64360.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60144.log 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T23:00:39.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61609.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61084.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60144.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40629.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61609.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61609.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40629.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42759.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40629.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42158.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43218.log: /var/log/ceph/ceph-client.admin.42759.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62049.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60882.log 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42158.log.gz 2024-09-17T23:00:39.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62049.log: 91.8% 0.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62049.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40181.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50712.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60882.log: /var/log/ceph/ceph-client.admin.40181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57393.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60882.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40181.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/objectstore_tool.50712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54804.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64108.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57393.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54804.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40505.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58079.log 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40505.log: /var/log/ceph/ceph-client.admin.64108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64108.log.gz 2024-09-17T23:00:39.146 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40505.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61480.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38856.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58079.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61480.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49697.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48215.log: /var/log/ceph/ceph-client.admin.38856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38856.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40180.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49697.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62175.log 2024-09-17T23:00:39.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41718.log: /var/log/ceph/ceph-client.admin.40180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40180.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60923.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60303.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62175.log: 95.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62175.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/objectstore_tool.50712.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60923.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62939.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42977.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60303.log: /var/log/ceph/ceph-client.admin.62939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39727.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60303.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62939.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42977.log.gz 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51193.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41397.log 2024-09-17T23:00:39.148 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39727.log: /var/log/ceph/ceph-client.admin.51193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63200.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39727.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51193.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41397.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50289.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59826.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63200.log: /var/log/ceph/ceph-client.admin.50289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63200.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50289.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59826.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52682.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62133.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48331.log: /var/log/ceph/ceph-client.admin.52682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63478.log 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52682.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-09-17T23:00:39.149 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63856.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62133.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58238.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63478.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63856.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63730.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58238.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57229.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63730.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.63142.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63730.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43991.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57229.log.gz 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63142.log: -- /var/log/ceph/ceph-client.admin.45355.log 2024-09-17T23:00:39.150 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63142.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43991.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45355.log: /var/log/ceph/ceph-client.admin.43057.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz/var/log/ceph/ceph-client.admin.50930.log: 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63688.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38677.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63688.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63240.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.38677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38677.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63240.log.gz 2024-09-17T23:00:39.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63562.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39532.log: /var/log/ceph/ceph-client.admin.53132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58707.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63562.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52108.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58707.log: /var/log/ceph/ceph-client.admin.52108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57471.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52108.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58707.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64276.log 2024-09-17T23:00:39.152 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57471.log: -- /var/log/ceph/ceph-client.admin.45275.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57471.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64276.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.39172.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64276.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45275.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64150.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.39172.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0%/var/log/ceph/ceph-client.admin.64150.log: -- replaced with /var/log/ceph/ceph-client.admin.39172.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64150.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52975.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52975.log.gz 2024-09-17T23:00:39.153 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63604.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63898.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63604.log: /var/log/ceph/ceph-client.admin.47032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63604.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61733.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64318.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63898.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61733.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63201.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56691.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63201.log: /var/log/ceph/ceph-client.admin.64318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64318.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63201.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62007.log 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56691.log.gz 2024-09-17T23:00:39.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40373.log 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58444.log 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40373.log: /var/log/ceph/ceph-client.admin.62007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62007.log.gz 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40373.log.gz 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62845.log 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40179.log 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58444.log.gz 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62845.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53591.log 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.40179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40179.log.gz 2024-09-17T23:00:39.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53591.log.gz 2024-09-17T23:00:39.156 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T23:00:39.172 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T23:00:39.190 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T23:00:39.420 INFO:teuthology.orchestra.run.smithi159.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T23:00:39.456 INFO:teuthology.orchestra.run.smithi159.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T23:00:40.416 INFO:teuthology.orchestra.run.smithi171.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T23:00:40.630 INFO:teuthology.orchestra.run.smithi171.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T23:00:42.702 INFO:teuthology.orchestra.run.smithi171.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T23:00:42.845 INFO:teuthology.orchestra.run.smithi159.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T23:00:43.049 INFO:teuthology.orchestra.run.smithi171.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T23:00:43.051 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-17T23:00:43.051 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m4.016s 2024-09-17T23:00:43.051 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m10.412s 2024-09-17T23:00:43.051 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.449s 2024-09-17T23:00:43.386 INFO:teuthology.orchestra.run.smithi159.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T23:00:44.859 INFO:teuthology.orchestra.run.smithi159.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T23:00:44.861 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-17T23:00:44.861 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m5.867s 2024-09-17T23:00:44.861 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m14.387s 2024-09-17T23:00:44.862 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m0.711s 2024-09-17T23:00:44.862 INFO:tasks.ceph:Archiving logs... 2024-09-17T23:00:44.862 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi159/log 2024-09-17T23:00:44.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T23:00:46.749 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi171/log 2024-09-17T23:00:46.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T23:00:48.041 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T23:00:48.050 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T23:00:48.050 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T23:00:48.053 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T23:00:48.137 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T23:00:48.145 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T23:00:48.146 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:00:48.148 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:00:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:00:48.163 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 -172us[ -168us] +/- 47ms 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -605us[ -605us] +/- 76ms 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +812us[ +812us] +/- 66ms 2024-09-17T23:00:48.167 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 -180us[ -181us] +/- 47ms 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -592us[ -592us] +/- 76ms 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +797us[ +797us] +/- 66ms 2024-09-17T23:00:48.169 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:00:48.169 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T23:00:48.178 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T23:00:48.178 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T23:00:48.208 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T23:00:48.293 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T23:00:48.368 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:00:48.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:00:48.400 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:00:48.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:00:48.429 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T23:00:48.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T23:00:48.483 INFO:teuthology.task.internal:Duration was 646.858954 seconds 2024-09-17T23:00:48.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T23:00:48.525 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T23:00:48.526 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T23:00:48.528 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T23:00:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:00:48.567 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:00:48.927 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T23:00:48.927 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-09-17T23:00:48.928 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T23:00:48.989 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-17T23:00:48.989 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T23:00:49.024 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T23:00:49.024 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T23:00:49.032 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T23:00:49.092 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T23:00:49.092 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T23:00:49.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T23:00:49.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T23:00:49.259 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T23:00:49.260 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T23:00:49.284 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T23:00:49.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T23:00:49.318 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:00:49.326 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:00:49.347 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-09-17T23:00:49.372 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-17T23:00:49.446 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:00:49.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:00:49.460 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:00:49.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:00:49.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T23:00:49.511 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T23:00:49.512 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi159 2024-09-17T23:00:49.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T23:00:49.552 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909554/remote/smithi171 2024-09-17T23:00:49.552 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T23:00:49.585 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T23:00:49.585 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T23:00:49.587 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T23:00:49.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T23:00:49.646 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T23:00:49.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T23:00:49.666 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T23:00:49.667 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T23:00:49.669 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T23:00:49.681 INFO:teuthology.orchestra.run.smithi159.stdout: 265903 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 23:00 /home/ubuntu/cephtest 2024-09-17T23:00:49.689 INFO:teuthology.orchestra.run.smithi171.stdout: 266088 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 23:00 /home/ubuntu/cephtest 2024-09-17T23:00:49.691 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T23:00:49.709 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T23:00:49.794 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 646.8589537143707 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T23:00:49.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T23:00:49.873 INFO:teuthology.run:pass